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50353-598 

DRIVING ASSIST SYSTEM 

FIELD OF THE INVENTION 

[01] The present invention relates to driving assist systems and methods for assisting 
efforts by operators to operate vehicles in traveling. 

DESCRIPTION OF THE BACKGROUND ART 

[02] There is a need for a driving assist system, which can reliably assist effort by an 
operator to operate a vehicle, such as, an automobile, in traveling. Currently available 
automobiles are equipped with a data acquisition system and a controller including a 
processor. Such data acquisition system can continuously acquire data including information 
on vehicle state, such as current vehicle velocity, and information on environment in a field 
around the vehicle. The information on environment includes the presence of obstacles, 
including a leading vehicle ahead. 

[03] Ideally, a driving assist system should avoid objection by the operator of the 
automobile in assisting the operator manual effort to reduce a risk of collision with an 
obstacle. However, if the vehicle should encoimter risky environment involving the presence 
of an obstacle, the driving assist system should avoid collision with the obstacle. Thus, there 
is a need for d system, which performs assist so that the operator can continue driving 
comfortably without any objectionable feel due to such assist. 

[04] JP-A 10-211886, published on August 11, discloses a steering control system, which 
modulates power-assist for avoidance of collision upon recognition of a risk within a field 
around the vehicle. This steering control system computes the magnitude of risk with respect 
to each of obstacles within the field around the vehicle. The magnitude of risk reflects the 
magnitude of impact upon collision with the obstacle. The magnitude of impact depends 
mainly upon the relative velocity between the vehicle and the obstacle. Accounting for this 

WDC99 867822-1.050353.0598 

SUBSTITUTE SPECIFICATION 



2 



fact, the known system computes an individual risk R(i) from an angular location a(i) 
expressed in terms of an azimuth angle of an obstacle i, a distance or separation d(i) between 
the subject vehicle and the obstacle i, and an odd function of the relative velocity Vref(i) 
between the subject vehicle and the obstacle i. This odd function involves a term of Vref(i)" 
(n = a real number grater than or equal to 3). The individual risk R(i) is expressed as, 

[05] R(i)=/M)±Aw(a(i)) Eq.l 
where, 

f|[Vrefl[i)) = Odd fimction that involves a term of VrefCi)" (n > 3); 

d(i) = Distance (or separation) between the subject vehicle and the obstacle i; 

k = Constant to adjust tiie distance d(i); and 

W(a(i)) = Weighting component applied to the obstacle i appearing in the angular location 
a . 

The individual risks are summed vertorially to produce a resultant risk R, which is expressed 
as, 

[06] R=2^(0 Eq.2 

f 

[07] The known steering control system is capable of avoiding collision with the obstacle 
by assisting the operator to operate the vehicle laterally to clear the obstacle. However, this 
system appears to be insufficient to avoid objectionable feel, which the operator might have. 
The operator may have such objectionable feel upon occurrence of a power assist, which has 
magnitude determined by the risk R, because a discrepancy exits between the risk R and a 
risk that is perceived by the operator. When the vehicle encounters an obstacle, the 
magnitude of risk R is proportional to the cube of the relative velocity between the vehicle 
and the obstacle, whereas the magnitude of risk perceived by the operator is proportional to 
the square of the relative velocity. 

[08] It is therefore an object of the present invention to provide a driving assist system for 
assisting effort by an operator to operate a vehicle in traveling, which can assist the operator 
without causing the operator to have any objectionable feel. 
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[09] It is another object of the present invention to provide a simple and economical 
driving assist system for assisting effort by an operator to operate a vehicle in traveling. 

SUMMARY OF THE INVENTION 

[10] The present invention provides, in one aspect thereof, a driving assist system for 
assisting effort by an operator to operate a vehicle in travehng, the driving assist system 
comprising: 

[11] a data acquisition system acquiring data including information on vehicle state and 
information on environment in a field around the vehicle; 

[12] a controller, mounted to the vehicle, for receiving the acquired data, for determining 
future environment in the field using the acquired data, for making an operator response plan 
in response to the determined fiiture environment, which plan prompts the operator to 
operating the vehicle in a desired manner for the determined fiiture environment, to 
determine command, and for generating the command; and 

[13] at least one actuator, mounted to the vehicle, for prompting the operator in response 
to the command to operating the vehicle in the desired manner. 

[14] The present invention provides, in second aspect thereof, a driving assist system for 
assisting effort by an operator to operate a vehicle in traveling, the driving assist system 
comprising: 

[15] a data acquisition system acquiring data involving information on the presence of a 
leading vehicle in a field around the vehicle; and 

[16] a controller, mounted to the vehicle, for receiving the acquired data, for determining a 

first extent to which the vehicles has approached to each other and a second extent to which 

the first extent might be influenced if a change in environment should occur, and for 

determining fiiture environment in the field based on the first and second extents. 

[17] The present invention provides, in third aspect thereof, a driving assist system for 

assisting effort by an operator to operate a vehicle in traveling, the driving assist system 

comprising: 
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[18] a data acquisition system acquiring data involving information on the presence of an 
obstacle in a field around the vehicle; 

[19] a controller, mounted to the vehicle, for receiving the acquired data, for determining a 
risk which the obstacle would cause the operator to perceive, for allocating commands for 
prompting the operator to operating the vehicle longitudinally and laterally, and for 
generating the commands; and 

[20] a plurality of actuators, mounted to the vehicle, to prompt the operator in response to 
the commands to operating the vehicle longitudinally and laterally. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[21] The present invention will be apparent fi-om reading of the following description in 
conjunction with the accompanying drawings. 

[22] Figure 1 is a control diagram showing control of a driving assist system. 
[23] Figure 2 illustrate parameters relevant to a leading and following traffic scene. 
[24] Figure 3 is a block diagram showing hardware of a first exemplary implementation of 
the present invention. 

[25] Figure 4 is a view illustrating how the hardware of the first implementation is 
arranged on an automobile. 

[26] Figure 5 is a block diagram showing hardware of a second exemplary implementation 
of the present invention. 

[27] Figure 6 is a view illustrating how the hardware of the second implementation is 
arranged on an automobile. 

[28] Figure 7 is a flow diagram illustrating the operation of the first implementation. 

[29] Figure 8 is a schematic view illustrating one example of ciurent environment in a 

field around the subject vehicle, which is determined by the first implementation. 

[30] Figure 9 is a schematic view illustrating a fiiture environment in the field around the 

subject vehicle, which is predicted by the first implementation, and the optimum path. 
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[31] Figure 10 is a timing diagram illustrating an example of a variation curve of the 
optimum steering angle for the vehicle to track the optimum path. 

[32] Figure 11 is a graphical representation of an example of steering reaction 
characteristic, which is provided at a moment of the curve of Figure 10, to prompt the vehicle 
operator to operating the steering wheel to the optimum steering angle. 

[33] Figure 12 is a flow diagram illustrating the operation of a second exemplary 
implementation of the present invention. 

[34] Figure 13 is a schematic view illustrating one example of current environment in a 
field around the subject vehicle, which is determined by the second implementation. 
[35] Figure 14 is a schematic view illustrating a future environment in the field around the 
subject vehicle, which is predicted by the second implementation. 

[36] Figure 15 is a timing diagram illustrating an example of a variation curve of the 
optimum vehicle velocity. 

[37] Figures 16A and 16B are timing diagrams illustrating the optimum coordinated 
variations of accelerator and brake pedal angular positions. 

[38] Figure 17 is a graphical representation of one example of accelerator pedal reaction 

characteristic, which is provided at a moment of the curve of Figure 16 A, to prompt the 

vehicle operator to operating the accelerator pedal to the optimum pedal position. 

[39] Figure 18 is a graphical representation of one example of brake pedal reaction 

characteristic, which is provided at a moment of the curve of Figure 16B, to prompt the 

vehicle operator to operating the brake pedal to the optimum pedal position. 

[40] Figure 19 is a schematic view of a servo motor of an accelerator reaction 

characteristic modulation actuator in operative relationship with an accelerator pedal. 

[41] Figure 20 is a block diagram showing hardware of a third exemplary implementation 

of the present invention. 

[42] Figure 21 is a view illustrating how the hardware of the third implementation is 
arranged on an automobile. 

[43] Figure 22 is a view illustrating an accelerator pedal reaction modulation mechanism. 
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[44] Figure 23 is a graph illustrating the relationship between accelerator pedal reaction 
force and stroke. 

[45] Figure 24 is a flow diagram illustrating the operation of the third implementation. 

[46] Figure 25 is a timing diagram illustrating how, in a traffic scene in Figure 2, a vehicle 

operator predicts a change in velocity of a leading vehicle. 

[47] Figure 26 is a IIP map. 

[48] Figure 27 is another RP map. 

[49] Figure 28 is a block diagram showing hardware of a fourth exemplary 
implementation of the present invention. 

[50] Figure 29 a view illustrating how the hardware of the third implementation is 
arranged on an automobile. 

[51] Figure 30 is a flow diagram illustrating the operation of the fourth implementation, 
[52] Figure 3 1 is a graphical representation of a function determining a period of time that 
is used for prediction of a future environment in a field aroimd the vehicle. 
[53] Figure 32 is a flow diagram illustrating the operation of a first embodiment of a fifth 
exemplary implementation of the present invention. 

[54] Figure 33 is a flow diagram illustrating the operation of a second embodiment of the 
fifth exemplary implementation of the present invention. 

[55] Figure 34 is a graphical representation illustrating the varying of risk perceived RP 

calculated as compared to risk actually perceived by operator during the first half and the 

second half when the vehicle is approaching to a leading vehicle. 

[56] Figure 35 is a graphical representation illustrating risks RPo and RPi. 

[57] Figure 36 is a map for parameters a, b. 

[58] Figure 37 is a graphical representation illustrating risks RPo, RPi, and RP2. 

[59] Figure 38 is a block diagram showing hardware of an eighth exemplary 

implementation of the present invention. 

[60] Figure 39 is a view how the hardware of the eighth implementation is arranged on an 
automobile. 
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[61] Figure 40 is a flow diagram illustrating the operation of the eighth implementation. 
[62] Figure 41 is a graphical representation of the varying of variance of separation for 
laser radar as compared to that for a CCD camera. 

[63] Figure 42 is a graphical representation of the varying of variance of relative velocity 
for the laser radar as compared to that for the CCD camera. 

[64] Figure 43 is a graphical representation of the varying of variance of separation for the 
CCD camera v^ith different kinds of obstacles. 

[65] Figure 44 is a graphical representation of the varying of variance of relative velocity 

for the CCD camera with different kinds of obstacles. 

[66] Figure 45 is an accelerator pedal reaction command (Fa) map. 

[67] Figure 46 is a brake pedal reaction command (Fb) map. 

[68] Figure 47 is a steering reaction command (Fs) map. 

[69] Figure 48 is a flow diagram illustrating the operation of a ninth exemplary 
implementation of the present invention. 

[70] Figure 49 is a map illustrating the relationship between RPk and operator inputs. 
[71] Figure 50 is a map illustrating the relationship between RPLongitudinai and an operator 
input for opening the vehicle longitudinally. 

[72] Figure 5 1 is a map illustrating the relationship between RPLaterai and an operator input 
for operating the vehicle laterally. 

[73] Figure 52 is an accelerator pedal reaction command (Fa) map. 
[74] Figure 53 is a brake pedal reaction command (Fb) map. 
[75] Figure 54 is a steering reaction command (Fs) map. 

[76] Figure 55 illustrates an accelerator pedal reaction characteristic according to the ninth 
implementation. 

[77] Figure 56 illustrates a brake pedal reaction characteristic according to the ninth 
implementation. 

[78] Figure 57 illustrates a steering reaction characteristic according to the ninth 
implementation. 
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[79] Figure 58 is a flow diagram illustrating the operation of a tenth exemplary 
implementation of the present invention. 

[80] Figure 59 is a gain map according to the tenth implementation. 
DETAILED DESCRIPTION OF THE INVENTION 

[81] Referring to Figure 1, a box 10 represents a vehicle operator, and a box 12 represents 
input devices. Input devices 12 represent a steering wheel, an accelerator pedal and a brake 
pedal, which a motor vehicle 14 is usually equipped with. 

[82] Vehicle operator 10 perceives the vehicle state through normal senses, represented by 
feedback line 16, and environment or situation in a field aroimd the vehicle through normal 
senses, represented by line 18. The operator appUes manual effort, represented by line 20, to 
the accelerator pedal, represented by input device 12 sending a power request command to an 
engine controller, not shown, of vehicle 14 through line 22. The operator applies manual 
effort, represented by line 24, to the brake pedal, represented by input device 12, sending a 
brake request command to a braking system, not shown, of vehicle 14 through line 26. The 
operator applies manual steering effort, represented by 28, to the steering wheel, represented 
by input device 12 sending a steering request command to a steering system, not shown, of 
vehicle 14 through line 30. Input device 12 provides reaction force to the manual effort by 
the operator. 

[83] The reference nxmieral 40 generally indicates a driving assist system for assisting 
effort by operator 10 to operate vehicle 14 in travelling. A data acquisition system (DAS) 42 
arranged on or off the vehicle, continuously acquires data including information on the state 
of vehicle 14 and information on environment in a field around the vehicle. The information 
on the vehicle state includes velocity Vf of vehicle 14. The information on envirormient 
includes the presence of obstacles, such as, a leading vehicle in the same lane and other 
adjacent vehicles in the same or other lanes, in the field around vehicle 14. DAS 42 provides 
the presence of each obstacle in terms of angular location of the obstacle with respect to the 
subject vehicle 14 and relative velocity Vr between the obstacle and the subject vehicle 14. 
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DAS 42 may provide velocity of each of the obstacles. The information on environment also 
includes the presence of lane markings in the field around vehicle 14. DAS 42 may provide 
the presence of lane markings in terms of lateral displacement of the subject vehicle 14. 
DAS 42 may provide weather condition in the way of the subject vehicle 14. 
[84] Driving assist system 40 receives and uses the acquired data, represented by line 44. 
A predicting module 46 uses the acquired data 44 to determine or predict future environment 
in the field around vehicle 14. Using the determined future environment as input 48, a plan 
making or building module 50 makes or builds an operator-response plan, which prompts 
vehicle operator 10 to operating vehicle 14 in a desired manner for the predicted future 
environment. To implement the operator-response plan, command or commands are 
determined. The determined command or commands are generated and provided to 
actuator(s) 52 through line 54. In response to the command(s), actuator(s) 52 prompts the 
vehicle operator 10 to operating the vehicle 14 in the desired manner. 

[85] In embodiments, the actuator(s) 52 acts on input device 12 through line 56 to 
modulate reaction characteristic to manual effort applied to the input device by the vehicle 
operator 10. In one embodiment, the future environment determined by predicting module 
46 involves a risk which each of the determined obstacles would cause the vehicle operator 
10 to perceive. Such risks are summed vertorially to produce a resultant risk. In this case, 
the planning module 50 makes an operator-response plan that prompts operator 10 to 
operating vehicle in a desired manner to reduce the resultant risk. In another embodiment, 
the future environment further involves a risk derived from the road condition. Such risk is 
determined by a lateral deviation of vehicle 14 from a lane and by curvature of the lane. In 
this embodiment, the planning module 50 makes an operator response plan that prompts 
operator 10 to operating vehicle in a desired manner to reduce the resultant risk 
superimposed by the risk derived from the road condition. In the embodiments, the operator- 
response plan made by planning module 50 includes the amount input to the actuator(s) 52. 
[86] Building of the operator response plan must account for type of actuator(s) 52 
employed. In one embodiment where the actuator(s) 52 is capable of modulating reaction 
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characteristic to manual steering effort by operator 10, plan making module 50 can make an 
operator-response plan, which prompts vehicle operator 10 to operating vehicle 14 laterally 
in such a manner as to reduce the resultant risk. In another embodiment where the actuator(s) 
52 is capable of modulating reaction characteristic to manual effort applied to the accelerator 
pedal along with modulating reaction characteristic to manual steering effort, plan making 
module 50 can make an operator-response plan, which prompts vehicle operator 10 to 
operating vehicle 14 longitudinally and laterally in such a manner as to reduce the resultant 
risk. 

[87] While the plan-making module 50 and predicting module 46 can be implemented in 
hardware or software, it is most likely implemented in software. In a software 
implementation, the modules 46 and 50 represent software instmctions stored in a memory of 
a computer, and executed in the processor, and the acquired data stored in memory. In 
hardware implementations, the modules 46 and 50 are implemented in digital logic, 
preferably in an integrated circuit. 

[88] Before describing implementations in detail, it is helpful to begin with consideration 
of parameters that define relationship between the two adjacent vehicles on the same lane. 
Figure 2 illustrates the parameters relevant to two adjacent vehicles in travel on the same lane 
at different velocities. In the parameters, time to contact TTC and time headway THW are 
described later. 

[89] The following description provides various implementations of the present invention. 

First Implementation of the Invention 

[90] Figure 3 provides hardware, and Figure 4 illustrates how the hardware is arranged on 
an automobile 70. 

[91] In Figure 3, the reference nimieral 40 A generally indicates a driving assist system. 
Driving assist system 40A includes an on board data acquisition system (DAS) 72. DAS 72, 
mounted to the vehicle 70, includes laser radar 74, a front camera 76, a rear right side (RRS) 
camera 78R, a rear left side (RLS) camera 78L, and a soiu-ce of vehicle speed 80. 
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[92] Referring to Figxire 4, vehicle 70 has laser radar 74 mounted to a front bumper 
thereof. Laser radar 74 may be mounted to a front grille of the vehicle. Laser radar 74 is of a 
type which can scan horizontally and laterally about 6 degrees to each side of an axis parallel 
to the vehicle longitudinal centerline, propagates infrared pulses forwardly and receives the 
reflected radiation by an obstacle, such as, a rear bumper of a leading vehicle. Laser radar 74 
can provide an angular location of the leading vehicle and a vehicle separation or a distance 
between the subject vehicle 70 and the leading vehicle. 

[93] Front camera 76 is of a CCD type or a CMOS type and mounted to vehicle 70 m the 
vicinity of the internal rear view mirror to acquire image data of a region in front of the 
vehicle. The region covered by front camera 76 extends from a camera axis laterally to each 
side by about 30 degrees. 

[94] RRS and RLS cameras 78R and 78L are, each, of a CCD type or a CMOS type. RRS 
camera 78R is moimted to vehicle 70 in the vicinity of the intemal rear right comer to 
acqxiire image data of rear scenery including the adjacent lane on the right-hand side. RLS 
camera 78R is mounted to vehicle 70 in the vicinity of the intemal rear left comer to acquire 
image data of rear scenery including the adjacent lane on the left-hand side. 
[95] Source of vehicle speed 80 may obtain a measure of vehicle velocity by processing 
outputs from wheel speed sensors. The source of vehicle speed 80 may include an engine 
controller or a transmission controller, which can provide a signal indicative of the vehicle 
velocity. 

[96] Driving assist system 40A includes a microprocessor-based controller 82. Controller 
82 receives the acquired data from DAS 72. From source of vehicle speed 80, controller 82 
receives information on the vehicle velocity Vf. From laser radar 74, controller 82 receives 
information on vehicle separation D between the subject vehicle 70 and the adjacent leading 
vehicle. From the image data provided by front camera 76, controller 82 receives 
information on vehicle separation D between the subject vehicle 70 and each of vehicles in 
front. From the image data provided by RRS and RLS cameras 78R and 78L, controller 82 
receives information on the presence of vehicles, in the adjacent lanes, approaching from the 
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rear, and it also receives information on extent to which each vehicle is approaching. From 
the image data provided by front camera 76, controller 82 receives information on the 
presence of lane markings on a road, information on the lateral position of the vehicle 
relative to the lane markings, and infomiation on the angular position of the vehicle relative 
to the lane markings. Using such information, controller 82 detemiines current environment 
or situation in a field around vehicle 70. Using the current and past values of environment, 
controller 82 predicts or determines future enviroimient in the field around vehicle 70 for 
building of an operator response plan. 

[97] According to this implementation, the operator response plan includes using the 
current and fiiture enviroimients to determine the optimum path for vehicle 70 to take, 
establishing the optimum steering angle for vehicle 70 to track the determined optimum path, 
and determining the optimimi steering reaction characteristic for prompting the vehicle 
operator to operating the steering wheel to the optimum steering angle. Controller 82 
determines the amoimt of command in response to the determined optimum steering reaction 
characteristic. Controller 82 generates the command and applied it to a steering reaction 
modulation actuator 84. 

[98] Steering reaction modulation actuator 84 includes a controller called a steering 
reaction characteristic modulator 86 and a servo motor 88. Servo motor 88 is incorporated in 
the steering mechanism to modulate steering reaction characteristic to operator steering 
effort. Modulator 86. controls output torque of servo motor 88 in response to the command 
from controller 82 such that the steering reaction characteristic is adjusted to the reaction 
characteristic determined at controller 82. 

[99] The flow diagram in Figure 7 illustrates a control routine of the first implementation 
of the present invention. The control routine, generally indicated at 100, is executed at 
regular time intervals of 50 milliseconds. 

[100] At input box 102, the processor of controller 82 inputs acquired data by DAS 72. 
Specifically, the processor inputs vehicle velocity Vf, an angular location of a leading vehicle 
and a vehicle separation D between the subject vehicle 70 and the leading vehicle, and 
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relative relation between the subject vehicle 70 and each of vehicles that are determined by 
RRS and RLS cameras 78R and 78L. 

[101] At box 104, using the input data at box 102, the processor determines current 
environment in a field around vehicle 70. Particularly, the processor builds a current hazard 
map visualizing the possibility of coUision of the subject vehicle 70 with each of other 
vehicles from current and past values of the acquired data at box 102. 

[102] The possibiUty of collision with another vehicle lowers as the vehicle 70 separates 
from it. This possibility may be expressed as a frmction of the reciprocal of vehicle 
separation or the reciprocal of the square of vehicle separation. If the relative velocity 
between the two vehicles can be neglected, isograms in the hazard map are concentric circles 
with the location of the vehicle as the center. However, if the relative velocity between the 
two vehicles is not negligible, the isograms protrude in the direction of a vector of the 
relative velocity. 

[103] Accounting for the vehicle lateral position from the lane marking and the road 
curvature has proven to be effective for enhanced building of a hazard map. The possibility 
of collision grows big as the vehicle gets close to the lane marking separating the adjacent 
lanes or to the edge of a road. Weighting component is the least when the vehicle follows the 
centerline of a lane, and it gets great as the vehicle is close the lane marking or to the edge of 
road. With different weighting components, this collision possibility due to the lane marking 
and the before-mentioned collision possibility due to other vehicles are summed to provide 
enhanced hazard map. The weighting component applied to the collision possibility due to 
other vehicles is less than the weighting component applied to the collision possibility due to 
the lane markings. 

[104] Figure 8 illustrates an example of a current environment expressed in terms of a 
hazard map at the moment (t = 0). The environment involves a vehicle traveling scenario on 
a road with three lanes. In the middle lane, the subject vehicle 70 is following a leading 
vehicle 120. Vehicle 70 is traveling at a vehicle velocity Vf, while vehicle 120 is traveling at 
a vehicle velocity Va. As Vf > Va, a vector originating at vehicle 120 indicates the relative 
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velocity between the two vehicles 70 and 120. In the left adjacent lane, a vehicle 122 is 
traveling at a vehicle velocity lower than Vf. A vector originating at vehicle 122 indicates 
the relative velocity between the two vehicles 70 and 122. In the right adjacent lane, a 
vehicle 124 is traveling at a vehicle velocity higher than Vf. 

[105] At the moment (t = 0), vehicle 122 closes a path 126 for vehicle 70 to take to the left 
adjacent lane. Thus, path 126 is not justified. Apparently, vehicle 124 is about to close 
space in the right adjacent lane, closing any path for vehicle 70 to take to the right adjacent 
lane. The vehicle separation between the two vehicles 70 and 120 allows vehicle 70 to stay 
in the middle lane. In this environment, what the vehicle operator has to do is to continue 
driving vehicle 70 in the middle lane. 

[106] At box 106, after it has determined the current environment, the processor predicts or 
determines fiiture environments related to different fixture moments. The different fiiture 
moments have different time values, respectively. The time values are arranged regularly 
with the largest time value being a predetermined time value of 5 seconds in this 
implementation. 

[107] Figure 9 illustrates an example of one fiiture environment expressed in terms of a 
hazard map at a fiiture moment (t =T). 

[108] At this fiiture moment (t = T), vehicle 122 opens a path 126 for vehicle 70 to take to 
the left adjacent although the vehicle separation between vehicles 70 and 120 has reduced 
and vehicle 124 has closed space in the right adjacent lane. In this fiiture environment, the 
path 126 is justified and may be set as the optimum path at the fiiture moment (t = T). 
[109] At box 108, the processor uses the predicted fiiture environments to determine the 
optimum path at each of the moments. The following provides a description on what has 
been taken into accoimt in determining the optimum path at each moment. The optimum 
path has been determined by finding, in a hazard map for each moment, a provisional 
location, within an area aroiind the location of vehicle 70, where the collision possibility is 
the local minimimi. The processor checks on one after another of the provisional location 
and the adjacent other locations whether or not lateral acceleration and yaw angular rate due 
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to a change from the current location to the one of the locations falls within an acceptable 
range. If the provisional location has past the test, it is used as the optimum location. If not, 
any one of the adjacent other locations that has past the text is used as the optimum location. 
[110] At box 1 10, the processor determines the optimum steering angle 8* to accomplish the 
optimum path, and the optimum steering reaction characteristic to prompt the vehicle 
operator to operating the steering wheel to the optimum steering angle 5*. The optimum 
steering angle 8* may be determined by the steering characteristic and the optimum path. 
Figure 10 provides an example of a variation curve of the optimirai steering angleS*. Figure 
1 1 provides one example of steering reaction characteristic, which is provided at a moment 
of the variation curve of Figure 10, to prompt the vehicle operator to operating the steering 
wheel at the optimum steering angle 6* at the moment. The steering reaction characteristic of 
Figure 1 1 provides almost zero reaction force (Fs = 0) at the optimum steering angle 5*. As 
the steering angle 5 moves to the right out of this optimum steering angle 6*, the steering 
reaction force Fs increases at slope angle of Ksr. As the steering angle 5 moves to the left 
out of this optimum steering angle 6*, the steering reaction force Fs increases at slope angle 
ofKsL. 

[Ill] The slope angles Ksr and Ksl are not fixed and determined by gradient of the 
collision possibiUty on one side in the vicinity of the optimum path and gradient on the 
opposite side in the vicinity thereof Such gradients can be given from isograms on both 
sides of the optimum path within the predicted hazard maps (see box 106 in Figure 7). 
Operator steering effort to increase deviation from the optimum path to left or right is 
opposed by a reaction force determined by slope angle Ksl or Ksr. If a rapid change in the 
collision possibility exits on the right side of the optimum path, the slope angle Ksr increases 
to provide increased reaction force. If a rapid change in the collision possibility exits on the 
left side of the optimum path, the slope angle Ksl increases to provide increased reaction 
force. This steering reaction characteristic prompts the vehicle operator to operating the 
steering wheel to track the optimum path. 
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[112] At output box 112, the processor outputs command indicative of the determined 
reaction characteristic to steering reaction modulation actuator 84 (see Figure 3). The 
command is appUed to the modulator 86 of the actuator 84. Under the control of the 
modulator 86, servo motor 88 operates to provide the determined steering reaction 
characteristic to the manual steering effort appUed to the steering wheel by the vehicle 
operator. 

Second Implementation of the Invention 

[113] Figure 5 provides hardware, and Figure 6 illustrates how the hardware is arranged on 
an automobile 70B. 

[114] In Figure 5, the reference nimieral 40A generally indicates a driving assist system. 
Driving assist system 40B includes an on board data acquisition system (DAS) 72B. DAS 
72B, mounted to the vehicle 70B, includes laser radar 74, a front camera 76, and a source of 
vehicle speed 80. DAS 72B is substantially the same as DAS 72 except the elimination of 
RRS and RLS cameras 78R and 78L. 

[115] Referring to Figure 6, vehicle 70B has laser radar 74 moxmted to a front bumper 
thereof Front camera 76 of a CCD type or a CMOS type is mounted to vehicle 70B in the 
vicinity of the internal rear view mirror to acquire image data of a region in front of the 
vehicle. Source of vehicle speed 80 may obtain a measure of vehicle velocity by processing 
outputs from wheel speed sensors. The soiu'ce of vehicle speed 80 may include an engine 
controller or a transmission controller, which can provide a signal indicative of the vehicle 
velocity. 

[116] In Figure 5, driving assist system 40B includes a microprocessor based controller 
82B. Controller 82B receives the acquired data from DAS 72B. From source of vehicle 
speed 80, controller 82B receives information on the vehicle velocity Vf. From laser radar 
74, controller 82B receives information on vehicle separation D between the subject vehicle 
70B and the adjacent leading vehicle. From the image data provided by front camera 76, 
controller 82B receives information on vehicle separation D between the subject vehicle 70B 
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and each of vehicles in front. From the image data provided by front camera 76, controller 
82B receives information on the presence of lane markings on a road, information on the 
lateral position of the vehicle relative to the lane markings, and information on the angular 
position of the vehicle relative to the lane markings. Using such information, controller 82B 
determines current environment or situation in a field around vehicle 70B. Using the current 
and past values of environment, controller 82B predicts or determines future environments in 
the field around vehicle 70B for building of an operator response plan. 

[117] According to this implementation, the operator response plan includes using the 
current and fixture environments to determine the optimum vehicle velocity for vehicle 70B 
to take, estabUshing the optimum pedal positions of an accelerator pedal and a brake pedal 
for vehicle 70B to travel at the determined optimimi vehicle velocity, and determining the 
optimimi accelerator and brake reaction characteristics for prompting the vehicle operator to 
operating the accelerator pedal and brake pedal to the optimum pedal positions. Controller 
82B determines the amount of accelerator pedal reaction command in response to the 
determined optimimi accelerator reaction characteristic. Controller 82B also detemiines the 
amoxmt of brake pedal reaction command in response to the determined optimum brake 
reaction characteristic. Controller 82B generates the accelerator reaction command and 
applies it to an accelerator reaction modulation actuator 140. Controller 82B generates the 
brake reaction command and applies it to a brake reaction modulation actuator 142. 
[118] Accelerator reaction modulation actuator 140 includes a controller called an 
accelerator pedal reaction characteristic modulator 144 and a servo motor 146. Brake 
reaction modulation actuator 142 includes a controller called a brake pedal reaction 
characteristic modulator 148 and a brake booster 150. 

[119] Referring also to Figure 19, servo motor 146 is incorporated into a link mechanism of 
an accelerator pedal 152 to modulate accelerator reaction characteristic to operator effort to 
depress accelerator pedal 152, Modulator 144 controls output torque of servo motor 146 in 
response to the command from controller 82B such that the accelerator reaction characteristic 
is adjusted to the reaction characteristic determined at controller 82B. 
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[120] Referring back to Figures 5 and 6, brake booster 150 can vary a brake assist to 
modulate brake reaction characteristic to operator effort to depress a brake pedal 154. 
Modulator 148 controls brake booster 150 in response to the command from controller 82B 
such that the brake reaction characteristic is adjusted to the reaction characteristic determined 
at controller 82B. 

[121] The flow diagram in Figure 12 illustrates a control routine of the second 
implementation of the present invention. The control routine, generally indicated at 160, is 
executed at regular time intervals of 50 milliseconds. 

[122] At input box 162, the processor of controller 82B inputs acquired data by DAS 72B. 
Using the image data from front camera 76, the processor determines the position of lane 
markings and the position of each of vehicles in front relative to the lane markings. Based on 
the relative position of each of the vehicles in front to the lane markings, the processor picks 
up a leading vehicle on the same lane as a target to be monitored by laser radar 74 for the 
subsequent control. The processor inputs an angular location of the leading vehicle, and a 
vehicle separation D between the subject vehicle 70B and the leading vehicle from laser 
radar 74. The processor receives vehicle velocity Vf from source of vehicle speed 80. 
[123] At box 164, using the acquired data received at box 162, the processor determines 
current environment in a field around vehicle 70B. Particularly, the processor builds a 
current hazard map visualizing the possibiUty of collision of the subject vehicle 70B with the 
following vehicle from current and past values of the acquired data. 

[124] The possibility of collision with the leading vehicle lowers as the vehicle 70B 
separates from it. This possibility may be expressed as a fimction of the reciprocal of vehicle 
separation or the reciprocal of the square of vehicle separation. If the relative velocity 
between the two vehicles can be neglected, isograms in the hazard map are concentric circles 
Avith the location of the vehicle as the center. However, if the relative velocity between the 
two vehicles is not negligible, the isograms protrude in the direction of a vector of the 
relative velocity. 
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[125] Figure 13 illustrates an example of a current environment expressed in terms of a 
hazard map at the moment (t == 0). The environment involves a vehicle traveling scenario 
within a lane on road. A leading vehicle 120 exists in the same lane. Vehicle 70B is 
traveling at a vehicle velocity Vf, while vehicle 120 is traveUng at a vehicle velocity Va. As 
Vf > Va, a vector originating at vehicle 120 indicates the relative velocity between the two 
vehicles 70B and 120. At the moment (t = 0), there is a sufficient vehicle separation. 
[126] At box 166, after it has determined the current environment, the processor predicts or 
determines future environments related to different future moments. The different future 
moments have different time values, respectively. The time values are arranged regularly 
with the largest time value being a predetermined time value of 5 seconds in this 
implementation. 

[127] Figure 14 illustrates an example of one future environment expressed in terms of a 
hazard m£^ at a future moment (t =T). At this future moment (t = T), the vehicle separation 
is not sufficient. 

[128] At box 168, the processor uses the predicted future environments to determine the 
optimum path at each of the moments. The following provides a description on what has 
been taken into account in determining the optimum path at each moment. The optimum 
path has been determined by finding, in a hazard map for each moment, a provisional 
location, within an area aroimd the location of vehicle 70B, where the collision possibility is 
the local minimum. The processor checks on one after another of the provisional location 
and the adjacent other locations whether or not longitudinal acceleration due to a change 
from the current location to the one of the locations falls within an acceptable range. If the 
provisional location has past the test, it is used as the optimum location. If not, any one of 
the adjacent other locations that has past the text is used as the optimum location. In the 
second implementation, the lateral control is out of consideration. Hence, the optimum path 
is found in the same lane so that it may be taken by vehicle 70B by varying the vehicle 
velocity. 
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[129] At box 170, the processor determines the optimum vehicle velocity V* to accompHsh 
the optimum path, the optimum accelerator and brake pedal positions 9a* and 9b*, and 
accelerator and brake pedal reaction characteristics. Figure 15 provides an example of a 
variation curve of the optimum vehicle velocity V*. Figures 16A and 16B provide one 
example of a coordinated variations of optimum accelerator and brake pedal positions ©a* 
and Ob* to accomplish the optimum velocity variation shown in Figure 15. In Figures 16A 
and 16B, during period of time Ti and T2, the optimum accelerator pedal position ©a* slightly 
reduces to gradually slow vehicle 70B down. During the subsequent period of time T2 and 
T3, the optimum accelerator pedal position 0a* has reduced to zero and the optimum brake 
pedal position Ob* increases temporally to effectively decelerate vehicle 70B. Finally, during 
period of time T3 and T4, the optimum accelerator pedal position ©a* accelerate vehicle 70B 
until it can beautifully follow the leading vehicle. 

[130] According to the second implementation, modulating accelerator and brake pedal 
reaction characteristic has accomplished the desired coordinated variations of the optimum 
accelerator and brake pedal positions ©a* and ©b*. 

[131] Figure 17 provides one example of accelerator pedal reaction characteristic, which is 
provided at a moment of the variation curve of Figure 16A, to prompt the vehicle operator to 
operating the accelerator pedal at the optimum accelerator pedal position ©a* at the moment. 
The accelerator reaction characteristic of Figure 17 is recessed from the usual reaction 
characteristic, as indicated by slope angle Kao, at the optimum accelerator pedal position ©a*. 
As accelerator pedal position 9a moves to the right out of this optimum accelerator pedal 
position 9a*, the accelerator pedal reaction force Fa increases at slope angle of Kaf. As 
accelerator pedal position ©Amoves to the left out of this optimum accelerator pedal position 
©A*, the accelerator pedal reaction force Fa increases at slope angle of Kab- 
[132] The slope angles Kaf and Kab are not fixed and determined by gradient of the 
coUision possibility if deviated longitudinally from the optimum position. Such gradients 
can be given from isograms around the optimimi position within the predicted hazard maps 
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(see box 166 in Figure 12). Operator manual effort to operate the accelerator pedal in a 
direction to increase longitudinal deviation from the optimum position is opposed by a 
reaction force determined by slope angle Kaf or Kab. If a rapid change in the collision 
possibility exits on the leading side of the optimum point, the slope angle Kaf increases to 
provide increased reaction force. If a rapid change in the collision possibility exits on the 
trailing side of the optimum point, the slope angle Kab increases to provide increased 
reaction force. This reaction characteristic prompts the vehicle operator to operating the 
accelerator pedal to track the optimum path. 

[133] Figure 18 provides one example of brake pedal reaction characteristic, which is 
provided at a moment of the variation curve of Figure 16B, to prompt the vehicle operator to 
operating the brake pedal at the optimum brake pedal position 0b* at the moment. The brake 
pedal reaction characteristic of Figure 18 is recessed from the usual reaction characteristic, at 
the optimum brake pedal position Ga*. As brake pedal position 9b moves to the right out of 
this optimum brake pedal position Gr*, the brake pedal reaction force Fb increases at slope 
angle of Kbf. As brake pedal position Gb moves to the left out of this optimum brake pedal 
position 0b*, the brake pedal reaction force Fb increases at slope angle of Kbb- 
[134] The slope angles Kbf and Kbb are not fixed and detemiined by gradient of the 
collision possibility if deviated longitudinally from the optimum position. Such gradients 
can be given from isograms around the optimum position within the predicted hazard maps 
(see box 166 in Figure 12). Operator manual effort to operate the brake pedal in a direction 
to increase longitudinal deviation from the optimxun position is opposed by a reaction force 
determined by slope angle Kbf or Kbb- If a rapid change in the collision possibility exits on 
the leading side of the optimum point, the slope angle Kbb increases to provide increased 
reaction force. If a rapid change in the collision possibility exits on the trailing side of the 
optimum point, the slope angle Kbf increases to provide increased reaction force. This 
reaction characteristic prompts the vehicle operator to operating the brake pedal to track the 
optimum path. 
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[135] At output box 172, the processor outputs first command indicative of the determined 
accelerator pedal reaction characteristic and second command indicative of the determined 
brake pedal reaction characteristic. The first command is applied to the modulator 144 of the 
actuator 140. Under the control of the modulator 144, servo motor 146 operates to provide 
the determined accelerator pedal reaction characteristic to the manual effort applied to the 
accelerator pedal by the vehicle operator. The second command is applied to the modulator 
148 of the actuator 142. Under the control of the modulator 148, brake booster 150 operates 
to provide the determined brake pedal reaction characteristic to the manual effort applied to 
the brake pedal by the vehicle operator. 

Third Implementation of the Invention 

[136] [First Embodiment] 

[137] Figure 20 provides hardware and Figure 21 illustrates how the hardware is arranged 
on an automobile 70C. 

[138] In Figure 20, the reference numeral 40C generally indicates a driving assist system. 
Driving assist system 40C includes an on board data acquisition system (DAS) 72C. DAS 
72C, mounted to the vehicle 70C, includes laser radar 74 and a source of vehicle speed 80. 
The laser radar 74 and source of vehicle speed 80 are moxmted to vehicle 70C in the same 
maimer as the second implementation (see Figures 5 and 6). 

[139] In Figure 20, driving assist system 40C includes a microprocessor based controller 
82C. Controller 82C receives the acquired data from DAS 72C in the same manner as 
controller 82B receives the acquired data firom DAS 72B. From source of vehicle speed 80, 
controller 82C receives information on the vehicle velocity Vf. From laser radar 74, 
controller 82B receives information on vehicle separation D between the subject vehicle 70C 
and the adjacent leading vehicle and relative velocity Vr between them. 

[140] Controller 82C determines, by calculation for example, a first extent to which vehicle 
70C has approached a leading vehicle ft-om vehicle separation D between the two vehicles 
and relative velocity Vr between them. Controller 82C determines current environment in a 
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field around vehicle 70C firom the first extent. Further, controller 82c predicts how the 
current environment will progress in fixture and uses the result of such prediction to output 
command to a controller called an accelerator pedal reaction characteristic modulator 144 of 
an accelerator reaction modulation actuator 140. Modulation actuator 140 includes a servo 
motor 146 and a stroke sensor 178. 

[141] Referring also to Figure 22, servo motor 146 and stroke sensor 178 are incorporated 
in a link mechanism of an accelerator pedal 152. Stroke sensor 178 senses the magnitude of 
operator manual effort appUed to accelerator pedal 152. The output of stroke sensor 178 is 
fed to modulator 144. In response to command from modulator 144, servo motor 146 
produces torque, modulating reaction force to the manual effort applied by vehicle operator 
to accelerator pedal 152. 

[142] Controller 82C receives information on vehicle separation D between vehicle 70C 
and a leading vehicle, information on relative velocity Vr between the vehicles, and 
information on vehicle velocity Vf of vehicle 70C from the acquired data by DAS 72C. 
Using the received information, controller 82C determines, by calculation for example, the 
first extent (=a first risk category) to which vehicle 70C has approached the leading vehicle, 
and a second extent (= a second risk category) to which vehicle 70C might be influenced due 
to a predictable change in movement of the leading vehicle. Controller 82C uses the first and 
second extents to predict fiiture environment expressed in terms of risk perceived (RP), 
determines an accelerator pedal reaction command AF based on the RP. Command AF is 
applied to modulator 144. Modulator 144 controls servo motor 146 in response to command 
AF, thus modulating reaction force versus stroke characteristic of accelerator pedal 152. 
[143] In Figure 23, a set of two parallel line segments intercomected by another set of two 
parallel line segments to enclose an area dotted illustrate usual reaction force characteristic. 
Under this condition, no additional force is applied to modulate the reaction force 
characteristic. With the accelerator pedal reaction modulation, this usual reaction force 
characteristic is elevated byAF over the whole range of stroke. AsAF is proportional to RP, 
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the vehicle operator can perceive through accelerator pedal 162 the current environment and 
predicted future environment in the field aroimd vehicle 70C. 

[144] The flow diagram in Figure 24 illustrates a control routine of the third 
implementation. The control routine, generally indicated at 180, is executed at regular time 
intervals of 50 milliseconds. 

[145] At input box 182, the processor of controller 82C inputs acquired data by DAS 72C. 
What are read at box 182 include velocity Vf of vehicle 70C, vehicle separation D between 
vehicle 70C and a leading vehicle, relative velocity Vr between the vehicles, and velocity Va 
of the leading vehicle. 

[146] At box 1 84, the processor calculates time to contact (TTC) and time headway (THW). 
[147] We introduced the notion of TTC to quantify a first extent to which the subject 
vehicle 70C has approached a leading vehicle in the traffic scene illustrated in Figure 2. This 
first extent is introduced to represent how much the vehicle 70C has approached the leading 
vehicle. 

[148] TTC is expressed as, 

[149] TTC = DA^r Eq. 3 

where, 

D = Vehicle separation between the vehicles in the traffic scene illustrated in Figure 2 
Vr = Relative vehicle velocity Vf - Va 

[150] In the traffic scene of Figure 2, most operators perceived risk and operated the 
following vehicles for deceleration well before TTC becoming less than 4 seconds. We 
considered that, risk, which each vehicle operator perceived, prompted the operator to 
operate the vehicle longitudinally to resume sufficient vehicle separation. The shorter TTC, 
the greater the magnitude of risk, which the leading vehicle causes the vehicle operator of the 
following vehicle to perceive, is. However, there is some discrepancy between TTC and risk 
perceived by the vehicle operator. Thus, TTC only is insufficient to quantify or express the 
risk perceived by operator. 

[151] The presence of such discrepancy may be confirmed by considering a traffic scene 
where the relative velocity Vr between the leading and following vehicles is zero. Li this 
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case, TTC is infinite irrespective of how short the vehicle separation D is. Apparently, the 
risk perceived by the operator of the following vehicle varies with different distances of 
vehicle separation D. The shorter the vehicle separation D, the bigger the risk perceived by 
the operator is. This is because the vehicle operator accounts for the magnitude of influence 
on TTC due to unpredictable drop in velocity Va of the leading vehicle. 

[152] With reference to Figure 25, this section provides a further description on the above 
issue. Immediately before the moment t = 0, the relative velocity Vr between the leading and 
following vehicles is positive and fixed, and TTC has been updated by calculating the 
equation 3. In operating the following vehicle, the operator keeps on predicting the 
magnitude of an impredictable change, particularly, a drop, in velocity Va of the leading 
vehicle, which might occur at the moment At after. Accurate prediction of Va at the moment 
At after is impossible, however. Thus, the operator tries to predict the velocity Va with 
considerable dispersion as illustrated in Figure 25. In Figure 25, if, at the moment At, the 
velocity Va drops to V2 fi-om Vi, TTC becomes shorter than expected At before, resulting in 
an increase in risk than expected. Predication of such increase in TTC fi-om the current and 
past values of TTC is difficult. 

[153] To overcome this difficulty, according to the third implementation, we introduced the 
notion of time headway (THW). THW is a physical quantity quantifying a second extent to 
which TTC might be influenced if a change in velocity Va of a leading vehicle should occur. 
The second extent is introduced to represent how much a change in relative velocity Vr, if 
any in immediate future, might influence TTC. THW is expressed as, 
[154] THW = DA^a Eq.4 
[155] or 

[156] THW = DA^f Eq. 5 

[157] THW is a measure of a timer that is set to count when the leading vehicle reaches a 
point on a road and will be reset subsequently when the following vehicle will reach the same 
point. The longer THW, the smaller the second extent is. That is, when THW is long, the 
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first extent is not greatly influenced due to a change in Va, if any, of the leading vehicle, so 
that a change in TTC is sufficiently small. 

[158] Using THW is to quantify the influence caused due to a future change in Va. In his 
respect, the use of THW determined by Va is recommendable rather than the other THW 
determined by Vf. Besides, the former represents the risk that is perceived by the operator 
with better accuracy than the latter does. However, if a value of Va that is given by a 
measure of Vf and a measure of Vr is less reliable than a value of Vf, the use of THW 
determined by Vf is recommendable. In the traffic scene where Va = Vf, the equations 4 and 
5 are equally recommendable. 

[159] With continuing reference to the flow diagram in Figure 24, after calculating TTC 
and THW at box 184, the logic goes to box 186. At box 186, the processor uses TTC and 
THW to create a physical quantity that is indicative of risk perceived (RP) by the vehicle 
operator. We introduced the notion of RP, which we call "Risk Perceived", to represent the 
magnitude of risk, which a vehicle operator would perceive upon predicting future 
environment in a field around the vehicle. In the traffic scene illustrated in Figure 2, the 
environment in the field around vehicle involves the presence of a leading vehicle. In 
creating the RP, we introduced the reciprocal of TTC (1/TTC) as the first extent and the 
reciprocal of THW (1/THW) as the second extent. According to the third implementation, 
they are summed to produce the RP, which is expressed as, 
[160] RP= a(l/THW) + b(l/TTC) Eq. 6, 

where b and a (b > a) are parameters weighting the first extent (1/TTC) and the second extent 
(1/THW), respectively, such that the second extent (1/THW) is less weighted than the first 
extent (1/TTC) is. The values of b and a are optimized after accounting for a statistics of 
values of THW and TTC collected in the traffic scene including leading and trailing vehicles. 
In this implementation, b = 8 and a = 1 . ^ 
[161] The equation 3 clearly reveals that time to contact TTC is a first risk category or 

component indicating how long it takes for a following vehicle to contact with a leading 

vehicle if it is assumed that the relative velocity Vr between the vehicles is constant. The 

equations 4 and 5 reveal that time headway THW is a second risk category or component 
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indicating how long it takes for the following vehicle to arrive at the point, which the leading 
vehicle has arrived at, from the instance of arrival of the leading vehicle if it is assumed that 
the relative velocity Vr will be subject to a drop. Using the first and second risk categories 
TTC and THW, defined by current values of D, Vf, Va and Vr, in the equation 6 yields risk 
perceived RP, which represents a predicted fixture environment in the field aroimd the 
vehicle. This operation is nothing but predicting a fiiture environment in a field around the 
vehicle using current values of D, Vf , Va and Vr. 

[162] The RP can provide quantitative expression as to how much a following vehicle has 
approached a leading vehicle continuously over a range from its separating from the leading 
vehicle to its approaching same. The more it increases, the more strongly an operator 
perceives risk of excessively approaching the leading vehicle in immediate fiiture. 
[163] The RP map in Figure 26 illustrates, within a THW- (1/TTC) plane, isograms of RP 
drawn based on the equation 6. The horizontal axis represents THW. The horizontal axis is 
a horizontal line originating at a value of 0.0 on the vertical axis and extending in the 
rightward direction. The vehicle is assumed to be following a leading vehicle with an 
unaltered relative velocity Vr. In Figure 26, the vehicle is operating at some operating point 
on the horizontal axis. The fiirther the operating point on the horizontal axis is distant from 
the vertical axis in the rightward direction, the fiirther the vehicle is separated from the 
leading vehicle. The vertical axis represens the reciprocal of TTC (1/TTC). Now, when the 
vehicle is assumed to start approaching the leading vehicle due to a drop in relative velocity 
Vr (= Vf - Va). In Figure 26, the operating point at which the vehicle is operating moves 
from the operating point on the horizontal axis as directed by an approaching motion vector. 
This motion vector is a resultant vector of a horizontal vector component oriented in the 
leftward direction and a vertical vector component in the upward direction. Next, when the 
vehicle is assumed to start separating from the leading vehicle due to an increase in relative 
velocity Vr. In Figure 26, the operating point at which the vehicle is operating moves from 
the operating point on the horizontal axis as directed by a separating motion vector. This 



WDC99 867822-1,050353.0598 



SUBSTITUTE SPECIFICATION 



28 

motion vector is a resultant vector of a horizontal vector component oriented in the rightward 
direction and a vertical vector component in the downward direction. 

[164] The operating point approaching the leading vehicle moves through an upper section 
of the isograms of RP in a direction increasing RP, while the separating point separating the 
leading vehicle moves through a lower or the remaining section of the isograms RP in a 
direction reducing RP. The upper and lower sections are interconnected at the horizontal 
axis. In the upper section, lines of the isograms extend from vertically spaced generally 
equidistant start points on the right edge of the RP map to differently spaced intermediate 
points on the horizontal axis, respectively. The intermediate points are arranged such that a 
distance between the adjacent two points of them becomes narrower and narrower as the 
THW becomes shorter and shorter. The lines of the isograms curve downwardly from the 
start points to the intermediate points to define a range of the highest or higher values of the 
RP. Such range is reached at a value of the reciprocal of TTC (1/TTC), which becomes 
smaller as the value of THW becomes smaller. Further, with the same value of THW, the 
higher the reciprocal of TTC (1/TTC), the higher the RP is. Besides, with the same value of 
the reciprocal of TTC (1/TTC), the shorter the THW, the higher the RP is. In the lower 
section, lines of the isograms extend from vertically spaced generally equidistant start points 
on a vertical line, not illustrated, near the left edge of the RP map to the differently spaced 
intermediate points on the horizontal axis and a point on the right edge of the map, 
respectively. The lines of the isograms curve upwardly from the start points to the 
intermediate points to define a range of the lowest or zero value of the RP. 
[165] With continuing reference to the flow diagram of Figure 24, after calculation of the 
RP at box 186, the logic goes to box 188. 

[166] At box 188, the processor uses the RP to determine an accelerator pedal reaction 

command AF, which is expressed as, 

[167] AF = K«RP Eq. 7 

where, 

K = Constant determined for adjustment. 
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[168] As readily seen from the RP map in Figure 26, the RP takes various values 
continuously over different values of THW and 1/TTC of environment in a space around the 
vehicle. The RP determines the command AF as expressed by the equation 7, which, in turn, 
determines accelerator pedal reaction force. This makes it possible for the operator to 
continuously know information how much the vehicle has approached to the leading vehicle. 
[169] At the next box 190, the processor outputs the command AF and applies it to the 
accelerator pedal reaction characteristic modulator 144 (see Figure 20). 
[170] The preceding description on the equation 6 and the RP map in Figure 26 clearly 
reveals that the RP takes various values continuously even if the first extent (1/TTC) and/or 
the second extent (1/THW) are subject to a change. Such smooth and continuous change in 
the RP makes it possible to vary accelerator pedal reaction force accordingly. Hence, the 
operator can recognize varying of environment through smooth and continuous change in 
accelerator pedal reaction force. 
[171] [Second Embodiment] 

[172] The second embodiment of the third implementation is substantially the same as the 
first embodiment thereof except the manner of producing the RP. In the first embodiment, 
the RP was expressed as the equation 6. In this second embodiment, RP is expressed as, 
[173] RP = max {a>THW, b/TTC} Eq. 8 

where, b and a (b > a) are parameters weighting the first extent (1/TTC) and the second 
extent (1/THW), respectively, such that the second extent (1/THW) is less weighted than the 
first extent (1/TTC) is. The values of b and a are optimized after accounting for a statistics of 
values of THW and TTC collected in the traffic scene including leading and trailing vehicles. 
In this implementation, b = 8 and a = 1 . 

[174] According to the equation, the larger one of a/THW and b/TTC is selected and used 
asRP. 

[175] The RP map in Figure 27 illustrates, within a THW- (1/TTC) plane, isograms of RP 
drawn based on the equation 8. The horizontal axis represents THW. The horizontal axis is 
a horizontal line originating at a value of 0.0 on the vertical axis and extending in the 
rightward direction. The vertical axis represents the reciprocal of TTC (1/TTC). 
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[176] Prior to a further description on the RP map in Figure 27, this section provides a brief 
review of the equation 6 and its RP map in Figure 26. When the vehicle is assumed to be 
separating from the leading vehicle, the relative vehicle velocity Vr is less than zero, and the 
term 1/TTC of the equation 6 is less than zero because TTC corresponds to DA^r- According 
to the equation 6, with the same THW, the RP takes various small values for different minus 
values of 1/TTC. Because the RP takes such small values, the accelerator pedal reaction 
command AF takes small values, accordingly. 

[177] According to the equation 8 and the RP map in Figure 27, the above-mentioned 
influence on the RP when the vehicle is separating from the leading vehicle is eliminated. 
This is because the equation 8 selects a/TPTW as RP when b/TTC is less than 0. With the 
same THW, the RP does not drop below a value that is determined by a current value of 
THW over different minus values of the reciprocal of 1/TTC. This provides a gradual 
reduction in RP when the vehicle is separating from the leading vehicle. 
[178] According to the third implementation, the RP is indicative of the magnitude of risk 
actually felt or perceived by an operator of a vehicle when, for example, involved in a traffic 
scene including a leading vehicle ahead. In the first embodiment, the RP is expressed by the 
equation 6 and illustrated in Figure 26. In the second embodiment, the RP is expressed by 
the equation 8 and illustrated in Figure 27. The RP determines the accelerator pedal reaction 
command AF as expressed by the equation 7. The command AF determines the magnitude of 
accelerator pedal reaction force to the manual effort applied to the accelerator pedal 152 by 
the operator. The equation 7 clearly indicates that the accelerator pedal reaction force is 
proportional to the RP. Thus, the accelerator pedal reaction force increases reflecting an 
increase in the risk actually perceived by the vehicle operator. The increase in accelerator 
pedal reaction force is transmitted to and felt by the operator through one's foot on the 
accelerator pedal 152, prompting the operator to allowing the accelerator pedal 152 to return 
to the released position. In other words, the operator is prompted to operating the vehicle 
longitudinally in a manner to reduce the risk, which the operator is caused to perceive by the 
leading vehicle. 
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[179] It will be appreciated, as an advantage, that the operator normally perceives the risk 
when the accelerator pedal reaction force is increased to prompt the operator to operating the 
vehicle in such a direction as to eliminate or at least reduce the risk. Accordingly, the 
operator is prompted to operating the vehicle in the direction with Httle objection to such 
driving assist. 

[180] Expressing in concrete terms, upon recognition of an increase in the RP by an 
increase in accelerator pedal reaction force, the operator is inspired to allow the accelerator 
pedal 152 to move toward the released position. The increase in accelerator pedal reaction 
force causes the operator foot to move automatically to release the accelerator pedal 152, 
prompting the operator to operating the vehicle in the direction to eliminate or at least reduce 
the risk. Besides, the increase in accelerator pedal reaction force discourages the operator 
from further depressing the accelerator pedal 152 to reduce the vehicle separation. 
[181] In the first embodiment of the third implementation, the equation 6 is used to produce 
RP. As is readily seem from the RP map in Figure 26, the RP varies continuously, so that the 
accelerator pedal reaction command AF and the accelerator pedal reaction force vary 
continuously, accordingly. Via such accelerator pedal reaction force, the operator is kept 
being informed of varying of environment in a field aroimd the vehicle with different values 
of 1/TTC andTHW. 

[182] In the second embodiment of the third implementation, the equation 8 is used to 
produce RP. As illustrated in Figure 27, the RP does not vary rapidly when the vehicle is 
separating from the leading vehicle where the values of 1/TTC are less than zero and very 
small. Because the RP does not vary rapidly, it is possible to control accelerator pedal 
reaction force, without any sudden change. 

[183] In the embodiments according to the third implementation, time to contact TTC and 
time headway THW are provided by calculation of simple equations involving easily 
measurable or obtainable physical quantities like velocity Vf of trailing vehicle, velocity Va 
of leading vehicle, and vehicle separation D. This works in suppressing an increase, if any, 
in number of component parts of a driving assist system. 
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[184] As has been described in connection with the equations 6 and 8, the parameter b is 
greater than the parameter a (b > a) such that the second extent (1/THW) is less weighted 
than the first extent (lATC) is. In the embodiments of the third implementation, the first 
extent (1/TTC) is heavily weighted than the second extent (1/THW) in producing the RP, 

Fourth Implementation of the Invention 

[185] Figure 28 provides hardware and Figure 29 illustrates how the hardware is arranged 
on an automobile 70D. 

[186] In Figure 28, the reference numeral 40D generally indicates a driving assist system. 
Driving assist system 40D is substantially the same, in hardware, as driving assist system 
40C except the provision of a firont camera 76 and a commxmication tool 208. Thus, the like 
reference numerals are used to indicate like parts or portions throughout Figures 20, 21, 28 
and 29. The fi-ont camera 76 and commimication tool 208 are parts of an on board data 
acquisition system (DAS) 72D mounted to the vehicle 70D. DAS 72D fiirther includes laser 
radar 74 and a source of vehicle speed 80. DAS 72D is substantially the same as DAS 72B 
shown in Figure 5 except the provision of communication tool 208 moimted to the vehicle 
70D. Communication tool 208 includes an infi-astructure receiver or a GPS receiver and 
receives information fi-om an antenna 210 attached to the vehicle 70D. In the same manner 
as the second implementation (see Figures 5 and 6), the laser radar 74, fi-ont camera 76, and 
source of vehicle speed 80 are moxmted to the vehicle 70D. 

[187] Driving assist system 40D includes a microprocessor based controller 82D. 
Controller 82D receives information on vehicle separation D between vehicle 70D and a 
leading vehicle, information on relative velocity Vr between the vehicles, and information on 
velocity Vf of vehicle 70D fi-om the acquired data by laser radar 74 and source of vehicle 
speed 80 of DAS 72D. Controller 82D also receives information on the leading vehicle after 
processing the image data captured by fi-ont camera 70. The processing of the image data 
includes filtering and image pattern recognition. Such processing may be carried out within 
or outside of controller 82D. Controller 82D receives current and fiiture information ahead 
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of vehicle 70D from communication tool 208. Such infomiation captured by commxmication 
tool 208 via antenna 210 will be later described. 

[188] Using the received information, controller 82D determines, by calculation for 
example, the first extent 1/TTC to which vehicle 70D has approached the leading vehicle. 
Controller 82D determines a period of time Tp in response to the current value of the first 
extent 1/TTC, and predicts a future value of the first extent the determined period of time 
after as a future environment in a field aroimd vehicle TOD. In this implementation, a risk 
perceived RP by operator expresses the future environment. Based on the RP, controller 82D 
determines an accelerator pedal reaction commandAF. CommandAF is applied to an 
accelerator pedal reaction characteristic modulator 144. Modulator 144 controls a servo 
motor 146 in response to commandAF, thus modulating reaction force versus stroke 
characteristic of an accelerator pedal 152. 

[189] The flow diagram in Figure 30 illustrates a control routine of the fovirth 
implementation. The control routine, generally indicated at 220, is executed at regular time 
intervals of 50 milliseconds. 

[190] At input box 222, the processor of controller 82D inputs acquired data by DAS 72D. 
What are read at box 222 include velocity Vf of the vehicle 70D, vehicle separation D 
between vehicle 70D and a leading vehicle, relative vehicle velocity Vr between the vehicles, 
and velocity Va of the leading vehicle. 

[191] At box 224, the processor calculates time to contact (TTC) that is expressed by the 
equation 3. As mentioned before in the third implementation, time to contact TTC is 
introduced to quantify a first extent to which the subject vehicle TOD has approached a 
leading vehicle in the traffic scene illustrated in Figure 2. As is readily seen from the 
equation 3, time to contact TTC indicates how much it takes for the subject vehicle to come 
into contact with a leading vehicle. When velocity Va is higher than velocity Vf, the relative 
velocity Vr is negative. In this case, time to contact TTC is negative. 
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[192] At box 226, the processor determines a period of time Tp, which indicates how many 
seconds to come prior to a future environment to be predicted, in response to the time to 
contact TTC. The period of time Tp is expressed as, 
[193] Tp = f(l/TTC) Eq.9 

This function involves, as a variable, the reciprocal of TTC. The fully drawn curve in Figure 
31 illustrates this function. As shown in Figure 31, the larger the reciprocal of TTC (1/TTC), 
the smaller the period of time z p is. 

[194] The period of time Tp corresponds to the magnitude of risk that is felt or perceived by 
the vehicle operator. In the case where time to contact TTC is negative or long so that the 
fu-st extent (1/TTC) is small, the subject vehicle 70D separates sufficiently from the leading 
vehicle. The current environment in the field around vehicle 70D does not require urgent 
attention by the operator. In such environment, the operator can afford to predict future 
environment long time to come to account for all available potential risk. Thus, the period of 
time Tp is long. In the case where time to contact TTC is short so that the first extent (1/TTC) 
is large, the subject vehicle TOD has approached the leading vehicle. The current 
environment in the field around vehicle 70D indicates the presence of risk in immediate 
future. In such environment, the operator will pay attention to the leading vehicle and cannot 
afford to predict future environment long time to come to account for all available potential 
risk. Thus, the period of time Tp is short. 

[195] With continuing reference to the flow diagram in Figure 30, after determining the 
period of time Tp at box 226, the logic goes to box 228. At box 228, the processor inputs the 
acquired data at commimication tool 208. What is read at box 228 includes; 

1. Operating state of a leading vehicle (such as, velocity Va and 
acceleration/deceleration) that is obtained through intervehicle communication; 
and 

2. Traffic condition and road condition in fi-ont of the subject vehicle, which are 
obtained through infi-astructure communication. 

[196] The use of intervehicle communication is limited to the case where the leading 
vehicle has intervehicle communication tool. 
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[197] At the next box 230, the processor inputs the acquired image data at front camera 76. 
The image data covering the front view of vehicle 70D is subject to image and pattern 
recognition for the processor to determine the presence of a leading vehicle and whether its 
rear brake lamps are turned "on" or "off. The information as to the state of the brake lamps 
is used later for predicting the magnitude of deceleration of the leading vehicle. 
[198] At box 260, the processor predicts deceleration XGa of the leading vehicle. If the 
leading vehicle is capable of performing intervehicle communication, the processor inputs 
the information on deceleration XGa of the leading vehicle out of the acquired data through 
the intervehicle communication. If the leading vehicle is not capable of performing the 
intervehicle communication, the processor predicts the deceleration XGa in the following 
manner. 

[199] First, the processor determines, out of the acquired image data from the front camera 
76, whether or not the brake lamps of the leading vehicle is turned "on". If this is the case, 
the processor measures, using a timer, for example, the elapse of time when the brake lamps 
are turned "on". Using the measure of the elapse of time and the variation of velocity Va of 
the leading vehicle, the processor predicts the deceleration XGa- For example, if the measure 
of the elapse of time is less than 0.5 seconds, the processor sets a predetermined value as a 
virtual deceleration value of the leading vehicle. If the measure of the elapse of time is not 
less than 0.5 seconds, the processor determines a predicted value by calculation out of the 
rate of variation of past values of velocity Va of the leading vehicle. 

[200] The predicted deceleration XGa of the leading vehicle is corrected accounting for the 
homogeneous degree of traffic flow. The homogeneous degree of traffic flow is predicted 
based on the traffic congestion of the road, which may be obtained, at box 228, out of the 
acquired data from the communication tool 208. The logic is such that the homogeneous 
degree of traffic flow is high when the traffic is congested and the variation of vehicle 
velocity Va is small. The processor corrects the predicted deceleration XGa in response to the 
predicted homogeneous degree of traffic flow. For example, the processor corrects the 
predicted deceleration XGa such that the predicted deceleration XGa decreases as the 
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predicted homogeneous degree of traffic flow increases. This is because a vehicle will not be 
accelerated or decelerated greatly when the road is congested and the homogeneous degree of 
traffic flow is high. 

[201] At the next block 234, the processor predicts fiiture value of the first extent (TTC) 
that would occur 7p after, using the current value of the first extent (TTC) obtained at box 
224, the period of time Tp determined at box 226, and the deceleration XGa predicted at box 
232. First, the processor calculates velocity of the subject vehicle TOD, velocity of the 
leading vehicle, and the vehicle separation between the vehicles. The processor calculates 
deceleration XGf of the subject vehicle TOD based on the current velocity Vf. When it is 
assxraied that deceleration XGf and XGa remains unaltered during the period of timcTp, future 
values of Vf, Va and D are expressed as, 

Vf(to+ T p) = V£(to) + XGf X r p Eq. 10 

Va(to+Tp)=Va(to)+XGaX Tp Eq. 1 1 

D(to+ T p) = D(to) - Vr X T p + (1/2) X (XGa -XGf) X r p^ Eq. 12 

Using the results given by calculations of the above equations 10-12, the processor predicts 
the fixture TTC, which is expressed as, 

TTC(toH- r p) = D(to+ r p)/{ Vf(to+ t p) - Va(to+ r p)} Eq. 1 3 
where, to is the current moment. 

[202] At box 236, the processor predicts future environment by calculating RP, which is 
expressed as, 

RP = b/ TTC(to+rp) Eq. 14 

[203] At box 238, the processor uses the RP to determine an accelerator pedal reaction 
command AF, which is expressed by the equation T. 
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[204] At the next box 240, the processor outputs the command AF and applies it to the 
accelerator pedal reaction characteristic modulator 144 (see Figure 28). 

[205] According to the fourth implementation, the period of time Xp is determined as a 
function of time to contact TTC. Time to contact TTC is determined based on Vf, Va, Vr and 
D. Using the information from the acquired data by front camera 76 and commimication tool 
208, the deceleration of a leading vehicle XGa is predicted. Using this deceleration XGa, a 
ftiture value of TTC that would occur Xp after is predicted. Risk perceived PR is determined 
as a function of the reciprocal of this future value of TTC. Finally, the RP is used to 
determine accelerator pedal reaction command AF. 

[206] According to the fourth implementation, a future value of extent to which vehicle 
70D might approach a leading vehicle is predicted from a current value of the extent. This 
futxire value of the extent corresponds to the real risk perceived or felt by the operator. In 
this implementation, RP (risk perceived) is indicative of this future value. The RP is 
predicted as a quantity indicative of future environment in a field around vehicle 70D. 
Accoxmting for the predicted RP, the accelerator pedal reaction control is performed. 
[207] The period of time Xp is used for prediction of a future value of RP. The period of 
time Xp is variable with the extent to which vehicle 70D has approached the leading vehicle. 
The faster the vehicle approaches the leading vehicle, the more immediate future is selected 
for the prediction. This has provided an accelerator pedal reaction control that corresponds 
to the magnitude of risk actually perceived or felt by operator. Like the previously described 
third implementation, the operator is continuously kept informed of how fast the vehicle is 
approaching the leading vehicle through the magnitude of reaction force felt during 
manipulation of accelerator pedal. 

[208] As different from the third implementation, the deceleration XGa of the leading 
vehicle, which is determined on the acquired data at front camera 76 and/or communication 
tool 208, is used. The use of deceleration XGa has enhanced prediction of future 
environment in a manner more closely corresponding to the magnitude of risk actually 
perceived by operator. 

WDC99 867822-1.050353.0598 

SUBSTITUTE SPECIFICATION 



38 

[209] In the fourth implementation, a current value and a future value of time to contact 
TTC are provided by calculation of equations involving, as variables, such physical 
quantities as vehicle speed and vehicle separation. This has lead to the minimal nimiber of 
new components needed for installing a driving assist system in a vehicle. Using 
communication tool 208 to receive information on deceleration XGa for use in the 
predication enhances the accuracy of a future value of TTC. 

[210] In the fourth implementation, both front camera 76 and communication tool 208 are 
used to determine deceleration XGa- The present invention is not limited to this example. 
Another example is the use of front camera 76 or commimication tool 208. Further example 
is the use of information provided by laser radar 74 or source of vehicle speed 80. A future 
value of TTC as determined using such deceleration XGa as predicted is satisfactorily 
reliable. 

[211] In the fourth implementation, laser radar 74 is used. Such laser radar may be replaced 
by millimeter radar or radar of other type. 

Fifth Implementation of the Invention 

[212] The fifth implementation is substantially the same as the third implementation except 
the provision of producing a selected one of different alert categories to be informed of via an 
appropriate alarm. Thus, the same technique as in the third and fifth implementations is used 
in determining TTC and THW. 

[213] The flow diagram of Figure 32 illustrates a control routine of a first embodiment of 
the fifth implementation. The control routine, generally indicated at 250, is executed at 
regular time intervals of 50 milliseconds. 

[214] The control routine 250 is substantially the same as the control routine 180 illustrated 
in Figure 24 in that boxes 252, 254 and 256 correspond to boxes 182, 184 and 186, 
respectively. A description on the boxes 252, 254 and 256 is hereby omitted for brevity. The 
following paragraph provides a description on two boxes 258 and 260 that follow box 256. 
[215] In Figure 32, at box 258, the processor of controller 82C (see Figure 20) selects one 
of alert categories in response to the RP. Various alert categories are stored for different 
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ranges in magnitude of RP. For example, alert category elevates as RP increases, in 
magnitude, into the adjacent higher range from a current range. Each alert category may 
contain the repetition rate of alarm such that the repetition rate of alarm elevates with 
elevation of alert category. At output box 260, the processor outputs the selected alert 
category to the alarm. 

[216] Setting alert category in response to the RP as described above will keep operator 
informed of the magnitude of risk when vehicle has approached a leading vehicle and/or 
when influence due to a predicted future change in environment is great. 
[217] In this embodiment, the RP expressed by the equation 6 has been used. Alternatively, 
RP expressed by the equation 8 or 14 may be used. An example using the equation 14 will 
be explained below. 

[2 1 8] The flow diagram of Figure 33 illustrates a control routine of a second embodiment of 
the fifth implementation. The control routine, generally indicated at 270, is executed at 
regular time intervals of 50 milhseconds. 

[219] The control routine 270 is substantially the same as the control routine 220 illustrated 
in Figure 30 in that boxes 272, 274, 276, 278, 280, 282, 284 and 286 correspond to boxes 
222, 224, 226, 228, 230, 232, 234 and 236, respectively. A description on the boxes 272, 
274, 276, 278, 280, 282, 284 and 286 is hereby omitted for brevity. The following paragraph 
provides a description on two boxes 288 and 290 that follow box 286. 

[220] In Figure 33, at box 288, the processor of controller 82D (see Figure 28) selects one 
of alert categories in response to the RP that has been determined at box 286. Various alert 
categories are stored for different ranges in magnitude of RP. For example, alert category 
elevates as RP increases, in magnitude, into the adjacent higher range from a current range. 
Each alert category may contain the repetition rate of alarm such that the repetition rate of 
alarm elevates with elevation of alert category. At output box 290, the processor outputs the 
selected alert category to the alarm. 
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[221] Setting alert category in response to the RP as described above will keep operator 
informed of the magnitude of risk when vehicle has approached a leading vehicle and/or 
when influence due to a predicted future change in environment is great. 
[222] In the embodiments of the fifth implementation, the repetition rate of alarm has been 
varied for different alert categories. The invention is not hmited to the repetition rate of 
alarm. Another example is to vary the volume and/or tone of alarm for different alert 
categories. Other example is to use various kinds of voice for different alert categories. 

Sixth Implementation of the Invention 

[223] The sixth implementation is substantially the same as the third, fourth and fifth 
implementations in the use of equation 6 in determining the RP. However, according to the 
sixth implementation, the use of equation 6 is limited to provide a new RP, which does not 
keep operator fi-om operating an automobile 70C (see Figures 20-24) approach to a leading 
vehicle when the vehicle separation D is sufficiently long. 

[224] The fiiUy drawn curve in Figure 34 illustrates the varying of RP expressed by the 
equation 6 with time t when the vehicle starts, at zero point (t = 0), approaching the leading 
vehicle at the rate of a very low approaching speed of 10 km/hour. For ease of 
understanding, it is assumed that, during the first half firom the zero point (t = 0) to a 
threshold moment (t = ta), the vehicle separation D between the vehicles is sufficiently long 
and the risk actually perceived or felt by the operator falls in a low risk level range. The 
second half begins immediately after the threshold moment (t = ta). During the second half, 
the vehicle separation D becomes shorter and shorter and the risk actually perceived by the 
operator grows with respect to time. 

[225] During the first half, paying much attention mainly to THW, the operator manipulates 
an accelerator pedal. During this first half, the RP expressed by the equation 6 and feU by the 
operator through the accelerator pedal grows in response to the reciprocal of THW, which the 
operator pays attention to. The risk actually perceived by the operator does not grow in 
response to the reciprocal of THW, however. Specifically, during the first half, the RP 
expressed by the equation 6 grows at the rate less than the rate at which the risk actually 
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perceived by operator grows. In Figure 34, the dotted line illustrates the varying of the risk 
actually perceived by operator during the first half and the subsequent transient to the second 
half. 

[226] According to the sixth implementation, a new risk perceived RPi is used to 
compensate for an insufficiency provided by the RP expressed by the equation 6. To avoid 
confusion, the RP expressed by the equation 6 is represented by a current risk perceived RPq. 
[227] Thus, the RPq and RPi are expressed as, 

RPo = RP = a(l/THW) + b(l/TTC) Eq. 15 

where b and a (b > a) are parameters weighting the first extent (1/TTC) and the second extent 
(1/THW), respectively, such that the second extent (1/THW) is less weighted than the first 
extent (1/TTC) is. The values of b and a are optimized after accounting for a statistics of 
values of THW and TTC collected in the traffic scene including leading and trailing vehicles. 
In this implementation, b = 8 and a = 1 . 

RPi = (c - THW) + (b/TTC) Eq. 16 

where b and c are parameters. In this implementation, b = 8 and c = 2.5. 

In this implementation, the threshold of THW is 0.5 seconds (THW = 0.5). When THW > 
0.5, the RPi is used. When THW <0.5, the RPq is used. The graph in Figure 35 illustrated 
this "hybrid" use of two RPq and RPi. In Figure 35, the one-dot chain line illustrates the 
variation of RPi against THW and the fiilly drawn curve illustrates the variation of RPq 
against THW. In this implementation, the RPi is used when THW > 0.5, and the RPo is used 
when THW <0.5. 

Seventh Implementation of the Invention 

[228] The seventh implementation is substantially the same as the sixth implementation. In 
this seventh implementation, another RP2 is used. The RP2 is expressed as, 

RP2 = d (c - THW) + (aA^HW) + (b/TTC) Eq. 17 
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where a, b, c and d are parameters. In this implementation, b = 8, and c = 2.5. 
d and a are determined as follows: 
whenTHW <0.5:d = 0, a=l 
whenTHW>0.5:d= l,a = a 

Apparently, the equation 17 provides the same performance as the equations 15 and 16 
provide. 

[229] Referring to Figure 36, an improved version using the equation 17 is described. 
According to this improved version, the parameters a and d used in the equation 17 are varied 
continuously in the illustrated pattern in Figure 36 to provide a smooth transition between 
two discrete RPo and RPi that exist in the equation 17. In Figure 36, the one-dot chain line 
illustrates the variation of parameter a with respect to THW, and the fully drawn line 
illustrates the variation of the parameter b with respect to THW. The fully drawn curve in 
Figure 37 illustrates the smooth varying of RPs when the parameters a and b are varied as 
illustrated in Figure 36. 

Eighth Implementation of the Invention 

[230] Figure 38 provides hardware and Figure 39 illustrates how the hardware is arranged 
on an automobile 70E. 

[231] In Figure 38, the reference numeral 40E generally indicates a driving assist system. 
Driving assist system 40E is a hybrid including an on board data acquisition system (DAS) 
72 and a steering reaction modulation actuator 84, which are substantially the same as their 
counterparts in the previously described driving assist system 40A (Figure 3), and an 
accelerator reaction modulation actuator 140 and a brake reaction modulation actuator 142, 
which are substantially the same as their counterparts in the previously described driving 
assist system 40B (Figure 5). ^ 
[232] In Figure 36, driving assist system 40E includes a microprocessor-based controller 
82E. 
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[233] Controller 82E recognizes the state of obstacles within environment in a field aroxind 
vehicle 70E by determining velocity Vf of vehicle 70E, relative position and relative velocity 
between vehicle TOE and each of other vehicles within the environment, and relative position 
between the adjacent lane marking and each of obstacles within the environment. Based on 
the recognized obstacle state, controller 82E determines risk which each of the recognized 
obstacles would cause the operator to perceive. Controller 82E divides each of the risks into 
a longitudinal risk component and a lateral risk component, and sums all of the longitudinal 
risk components to give a total longitudinal risk and all of the lateral risk components to give 
a total lateral risk. Based on the total longitudinal and lateral risks, controller 82E determines 
longitudinal commands and a lateral command. 

[234] Controller 82E outputs the longitudinal commands for application to an accelerator 
pedal reaction characteristic modulator 144 and a brake pedal reaction characteristic 
modulator 148, respectively. In response to the applied longitudinal commands, the 
accelerator pedal reaction characteristic modulator 144 and the brake pedal reaction 
characteristic modulator 148 control servo motor 146 and brake booster 150, thereby 
modulating the accelerator pedal and brake pedal reaction force characteristics. Modulating 
the accelerator pedal and brake pedal reaction characteristics prompt the vehicle operator to 
manipulating an accelerator pedal 152 and a brake pedal 154 to appropriate positions, 
respectively. 

[235] Controller 82E outputs the lateral command for application to a steering reaction 
characteristic modulator 86. In response to the applied lateral command, the steering 
reaction characteristic modulator 86 controls servo motor 88. Modulating the steering 
reaction characteristic prompts the vehicle operator to manipulating a steering wheel to an 
appropriate angular position. 

[236] The flow diagram in Figure 40 illustrates a control routine of the eighth 
implementation of the present invention. The control routine, generally indicated at 310, is 
executed at regular time intervals of 50 milliseconds. 

WDC99 867822-1.050353.0598 

SUBSTITUTE SPECIFICATION 



44 

[237] At input box 312, the processor of controller 82E inputs acquired data by DAS 72. 
Specifically, the processor inputs vehicle velocity Vf, an angular location of a leading vehicle 
and a vehicle separation D between the subject vehicle 70E and the leading vehicle. The 
processor also inputs relative position of the vehicle 70E to the adjacent lane marking out of 
the image data from the front camera 76. The relative position of the vehicle 70E to the 
adjacent lane marking includes a lateral position and an angular position of the vehicle 70E 
with respect to the lane marking. Out of the image data from the front camera 76, the 
processor further inputs the shape of the lane marking. Out of the image data from the front 
camera 76, the processor still ftirther inputs an angular location of the leading vehicle and a 
vehicle separation D between the vehicle 70E and the leading vehicle. Out of the image data 
from RRS and RLS cameras 78R and 78L, the processor inputs an angular location of each 
of vehicles existing in the adjacent lane(s) and a vehicle separation between the vehicle 70E 
and each of the vehicles. The processor also input velocity of the vehicle 70E from the 
source of vehicle speed 80. Besides, the processor recognizes the presence of and kinds of 
obstacles around the vehicle 70E out of the image data from the front camera 76 and the 
image data from RRS and RLS cameras 78R and 78L. Recognizing the kinds of obstacles 
includes pattem recognition to identify them whether they are a four wheel vehicle or a two 
wheel vehicle or a pedestrian. 

[238] At box 314, the processor determines current environment in the field around the 
vehicle 70E in terms of position of each of the obstacles/vehicles relative to the vehicle 70E, 
direction of movement of each of the obstacles relative to the vehicle 70E, and speed at 
which each of the obstacles relative to the vehicle 70E. Specifically, the processor 
determines current values of position, direction and speed of each of the obstacles/vehicles 
from the past stored values thereof that were determined in the previous cycles and the input 
data at box 312. 

[239] At box 316, using the data on current environment determined at box 314, the 
processor calculates time to collision TTC between the vehicle 70E and each of the obstacles. 
TTCk indicates TTC between the vehicle 70E and an obstacle k. TTCk is expressed as, 
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where 

Dk = Separation (or distance) between the vehicle 70E and an obstacle k; 
Vrk = Relative velocity between the vehicle 70E and the obstacle k; 
'a(Dk) = Variance of separation; and 
<^(Vrk) = Variance of relative velocity. 

[240] The following several paragraphs provide a description on the variances a(Dk) and 
a(Vrk). 

[241] An error in data might grow should if a sensor or a camera be subject to uncertainty 
or unforeseeable event. Accoimting for how much such an error might be, variance of 
separation a(Dk) and variance of relative velocity a(Vrk) are determined depending upon 
what an obstacle k is and which sensor or camera is used to recognize the obstacle k. 
[242] For example, the laser radar 74 is superior to a CCD camera, which is used as the 
front camera 76 and RRS and RLS cameras 78R and 78L, in its capability of providing 
accurate measure in separation or distance between two vehicles irrespective of how far they 
are separated. The curves in Figure 41 illustrate the variances of separation a(Dk) set for 
laser radar 74 and CCD camera, respectively. The CCD camera is used for front camera 76 
and RRS and RLS cameras 78R and 78L. As shown in Figure 41, when laser radar 74 is 
used to measure a separation Dk between the vehicle 70E and an obstacle k in the form of an 
automobile, variance a(Dk) is kept constant over any different values in the separation to be 
measxu-ed. When the CCD camera is used to measure the separation Dk, variance a(Dk) 
increases exponentially. When the separation Dk is less than a predetermined value, the CCD 
camera provides better accuracy than laser radar 74 does, and variance a(Dk) for CCD 
camera is less than that for laser radar 74. 

[243] The curves in Figure 42 illustrate the variances of relative velocity aCVrk) set for laser 
radar 74 and CCD camera, respectively. As shown in Figure 42, when laser radar 74 is used 
to measure relative velocity Vrk between the vehicle 70E and an obstacle k in the form of an 
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automobile, variance a(Vrk) increases linearly as the relative velocity Vrk increases. When 
the CCD camera is used to measure the relative velocity Vrk, variance a(Vrk) increases 
exponentially. 

[244] Using pattem recognition technique, image data from front camera 76 and RRS and 
RLS cameras 78R and 78L are analyzed by the processor to determine what an obstacle k is. 
A family of curves in Figure 43 illustrates the variance a(Dk) set for CCD camera. A family 
of curves in Figure 44 illustrates the variance aCVrk) set for CCD camera. A four wheel 
vehicle, a two wheel vehicle, a pedestrian and lane markings were selected and used as an 
obstacle k for measurement by the CCD camera. 

[245] When the CCD camera is used to measure the separation Dk, the bigger the obstacle 
k, the higher the accuracy of measurement is. Accordingly, as shown in Figure 43, the 
variancea(Dk) for measuring separation from a four wheel vehicle is less than that for 
measuring separation from two wheel vehicle or pedestrian. 

[246] When the CCD camera is used to measure the relative velocity Vrk, the variance 
a(Vrk) is determined depending on the magnitude of relative velocity which may be assumed 
for the kinds of obstacle k, respectively. As shown in Figure 44, the higher the assumed 
relative velocity, the larger the variance a(Vrk) is. Comparing relative velocity assumed for 
four wheel vehicle to that assumed for two wheel vehicle or that assumed for pedestrian, the 
former is higher than the latter. Accordingly, the variance a(Vrk) for four wheel vehicle is set 
greater than that for two wheel vehicle or that for pedestrian. 

[247] In Figure 43, the variance a(Dk) for lane marking is set less than those for four wheel 
vehicle, two wheel vehicle and pedestrian. In Figure 44, the variance a(Vrk) for lane marking 
is set less than those for four wheel vehicle, two wheel vehicle and pedestrian. 
[248] With reference again to the flow diagram in Figure 40, after calculating TTCk at box 
316, the logic goes to box 318. At 318, the processor uses TTCk to calculate risk perceived 
RPk, which an obstacle k would cause operator to perceive. RPk is expressed as, 

RPk=— ^xW. Eq.l9 
TTCk ^ 
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where, 

Wk = Weighting component applied to the obstacle k. 

Various values are set as weighting component for different obstacles in kind. For example, 
when a four wheel, a two wheel vehicle or a pedestrian is an obstacle k, the operator would 
be influenced most upon approaching such obstacle k. Thus, in this implementation, Wk = 1, 
when obstacle k is one of a four wheel, a two wheel vehicle and a pedestrian. When lane 
marking is an obstacle k, the operator would be influenced less upon approaching the lane 
marking. Thus, in this implementation, Wk = about 0.5, when obstacle k is lane marking. 
Operator would be influenced less upon approaching lane marking beyond which lies a 
railroad or a wall or a fence than one would be influenced upon approaching lane marking 
beyond lies the adjacent lane. Thus, different values may be set as Wk depending upon 
where the lane marking exists. 

[249] Lane markings are distributed within a range of angular locations ahead of the vehicle 
70E. Risk with regard to lane markings is determined by integrating RPk over the whole 
range of angular locations. Risk perceived RPlane is expressed as. 



where 

Wiane = Weighting component 

[250] With continuing reference to Figure 40, at box 320, the processor sums longitudinal 
components of individual risk perceived RPk including risk with regard to lane markings 
RPlane to providc a total longitudinal risk RPLongimdinai. The total longitudinal risk RPLongitudinai 
is expressed as, 

RPumgitudinal = ^RPkCOSa Eq. 21 




Eq. 20 



where. 
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= Angular location of an obstacle k as viewed from the vehicle 70E 5k = 0 when the 
obstacle k is located in front of the vehicle, ^k = 1 80 when the obstacle k is located behind 
the vehicle. 

[251] At the next block 322, the processor sums lateral components of individual risk 
perceived RPk including risk with regard to lane markings RPiane to provide a total lateral risk 
RPLaterai. The total lateral risk RPLaterai is expressed as, 

RPutenii = 2] ^* sinft Eq. 22 

k 

[252] At box 324, the processor determines longitudinal commands by referring to the 
relationship shown in Figure 45 and the relationship shown in Figure 46 using the total 
longitudinal risk RPLongimdmai- One of the longitudinal commands, namely, an accelerator 
pedal reaction conmiand Fa, is provided to modulate an accelerator pedal reaction 
characteristic. The other command, namely, a brake pedal reaction command Fb, is provided 
to modulate a brake pedal reaction characteristic. They provide such a relationship that, 
when the total longitudinal risk RPLongimdinai increases, the accelerator pedal reaction 
command Fa increases, prompting operator to releasing the accelerator pedal 152, while the 
brake pedal reaction force Fb decreases, prompting the operator to depressing the brake pedal 
154. 

[253] The fiiUy drawn line in Figure 45 illustrates the relationship between the accelerator 
pedal reaction command Fa and the total longitudinal risk RPLongitudinai- As will be readily 
seen from Figure 45, the command Fa remains zero when RPumgitudinai is less than a first 
threshold RPq. When RPLongitudinai exceeds the first threshold RPo but it is still less than a 
second threshold RPmax that is greater than the first threshold RPq, the command Fa is 
proportional to RPLongitudinai. The command Fa takes the maximum when RPtongitudinai 
exceeds the second threshold RPmax- 

[254] The fully drawn line in Figure 46 illustrates the relationship between the brake pedal 
reaction command Fb and the total longitudinal risk RPLongitudinai- When RPLongitudinai is 
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greater than the second threshold RP„u«, the greater longitudinal risk RPLongitudinai, the less the 
command Fb is. When RPtongitudinai is greater than a third threshold RPi that is greater than 
the second threshold RPmax, the command Fb is at the minimum FBmin, holding the brake 
pedal reaction force to the minimum. When RPLongitudinai is not greater than the second 
threshold RPmax, the conunand Fb is zero, leaving the brake pedal reaction characteristic 
unaltered. 

[255] With reference to Figures 45 and 46, when the longitudinal risk RPLongitudinai is not 
greater than the threshold RPmax, the accelerator pedal reaction characteristic is altered, 
letting operator know the magnitude of longitudinal risk RPLongitudinai by means of a change in 
reaction force of the accelerator pedal. When the longitudinal risk RPLongitudinai exceeds the 
threshold RPmax, the accelerator pedal reaction force is maximized, prompting the operator to 
releasing the accelerator pedal. At the same time, the brake pedal reaction force is reduced, 
allowing the brake pedal to be depressed easily when the operator steps on the brake pedal. 
[256] In Figure 40, at box 326, the processor determines a lateral command by referring to 
the relationship shown in Figure 47 using the total lateral risk RPLaterai- The lateral command, 
namely, a steering reaction command Fs, is provided to modulate a steering reaction 
characteristic. It provides such a relationship that, when the total lateral risk RPLateraii 
increases, the steering reaction command Fs increases in a manner to prompt operator to 
steering the vehicle to reduce the risk RPuterai. The fully drawn line in Figure 47 provides 
the relationship between the total lateral risk RPuterai and the steering reaction command Fs. 
In Figure 47, the total lateral risk RPLaterai is positive when risk on the right hand side exceeds 
that on the left hand side, but it is negative when risk on the left hand side exceeds than on 
the right hand side. 

[257] As shown in Figure 47, the steering reaction command Fs ranges from 0 to a 
maximum right-tum reaction Fsmax when the total lateral risk RPuterai is greater than 0 but 
less than a right-tum threshold RPmax- When the total lateral risk RPuterai is in the vicinity of 
zero, the command Fs is zero, leaving steering reaction characteristic unaltered. 
Subsequently, when the total lateral risk RPuterai increases, the command Fs increases with 
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the increasing total lateral risk RPuterai- When the total lateral risk RPLaterai is equal to or 
greater than the right-turn threshold RPmax, the conunand Fs stays at the maximum right-turn 
reaction Fsxnax. As the conmiand Fs increases from zero, the reaction force to operator right 
turn steering effort increases from a level of the usual steering reaction characteristic. 
[258] With continuing reference to Figure 47, the steering reaction command Fs ranges 
from 0 to a maximum left-turn reaction -Fsmax when the total lateral risk RPLaterai is less than 
0 but greater than a left-tum threshold -RPmax. When the total lateral risk RPLaterai is in the 
vicinity of zero, the command Fs is zero, leaving steering reaction characteristic unaltered. 
Subsequently, when the total lateral risk RPLaterai decreases, the command Fs decreases with 
the decreasing total lateral risk RPLaterai. When the total lateral risk RPuterai is equal to or less 
than the left-tum threshold -RPmax, the command Fs stays at the maximimi left-tum reaction - 
Fsmax. As the command Fs decreases from zero, the reaction force to operator left turn 
steering effort increases from a level of the usual steering reaction characteristic. 
[259] With reference again to Figure 40, at output box 328, the processor outputs the 
commands determined at boxes 324 and 326 using the illustrated relationships in Figures 45, 
46 and 47. These commands include the accelerator pedal reaction command Fa, the brake 
pedal reaction command Fb, and the steering reaction command Fs for application to the 
accelerator pedal reaction characteristic modulator 144, brake pedal reaction characteristic 
modulator 148 and steering reaction characteristic modulator 86 (see Figure 38), 
respectively. 

[260] The preceding description on the flow diagram in Figure 40 clearly indicates that the 
controller 82E accounts for longitudinal and lateral risk components calculated based on 
individual risk derived from each of obstacles within a field aroxmd the vehicle 70E. In 
response to the longitudinal and lateral risk components, the controller 82E allocates 
commands Fa, Fb and Fs for prompting operator to operating the vehicle 70E longitudinally 
and laterally in an appropriate manner to cope with such risk components. This may be 
readily confirmed by referring to the description on boxes 320, 322, 324 and 326. In the 
eighth implementation of the invention, this allocating fimction is implemented in terms of 
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software (instructions) component. However, the invention is not limited to this. The 
allocating function may be implemented in terms of software and hardware components or 
hardware component. 

[261] In the eighth implementation, the controller 84E determines the magnitudes of 
accelerator pedal reaction command Fa, brake pedal reaction command Fb, and steering 
reaction command Fs based on the total longitudinal risk RPLongitudinai and total lateral risk 
RFLaterai. In this implementation, the magnitudes of such commands are determined using 
the relationships illustrated in Figures 45, 46 and 47. These illustrated relationships show 
only one example of an operator response plan that prompts operator to operating the vehicle 
70E in a manner to negotiate with the longitudinal and lateral risks RPLongitudinai and RPLaterai- 
In this example, the accelerator pedal reaction characteristic, brake pedal reaction 
characteristic and steering reaction characteristic are modulated or altered to prompt the 
operator to operating the vehicle 70E in a direction toward where the longitudinal and lateral 
risks RPLongitudinai and RPLaterai are lowcrcd. 

[262] With reference to Figures 38 to 47, the eighth implementation of the present invention 
has been described. This implementation provides features as follows. 

[263] (1) As information on environment in a field around the vehicle 70E, the controller 
82E detemiines the presence of all of obstacles, including a leading vehicle, a trailing 
vehicle, a vehicle in the adjacent lane, a pedestrian, and lane markings. The controller 82E 
calculates, for each of the obstacles, a risk RPk. Accounting for the risks RPk from all 
directions of the vehicle 40E, the controller 82E allocates commands Fa, Fb, and Fs for 
prompting the vehicle operator to operating the vehicle 40E longitudinally and laterally in a 
direction appropriate to cope with the available risks RPk. It will be appreciated that this 
implementation provides an appropriate assist to the vehicle operator. 

[264] (2) The controller 82E provides an appropriate assist to manual effort to operate the 
accelerator pedal by modulating or altering accelerator pedal reaction in response to the 
determined command Fa- 
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[265] (3) For each of the obstacles, the controller 82E calculates TTCk by dividing 
separation by relative velocity Vrk, and determines risk RP^ as a function of the reciprocal 
of TTCk. This risk RPk represents an extent how quickly the vehicle 70E has approached the 
obstacle k. This implementation provides an appropriate assist to the vehicle operator to 
opening the vehicle 70E longitudinally and laterally in response to the extent how quickly the 
vehicle 70E has approached the obstacle k. 

[266] (4) As is indicated by equation 18, TTCk = ^* ~ , determining TTCk involves 

variance of separation a(Dk) and variance of relative velocity a(Vrk). In use of laser radar 74, 
the variances a(Dk) and a(Vrk) are subject to the illustrated varying in Figures 41 and 42. In 
use of CCD camera, the variances a(Dk) and a(Vrk) are subject to the illustrated varying in 
Figures 43 and 44. Employing the variances a(Dk) and a(Vrk) in determining TTCk allows 
use of diverse radars and cameras in on board DAS 72. 

[267] (5) As is indicated by equation 19, RPk = — — xfF*, determining RPk involves 

TTCk 

weighting component Wk. The weighting component Wk takes an appropriate value that has 
been selected out of various values predetermined for different kinds of obstacles and for 
different surroundings of lane marking. Employing the weighting component Wk in 
determining RPk allows diverse obstacles to be processed in determining commands Fa, Fb 
and Fs. 

[268] (6) With respect to the vehicle 70E, angular locations of all of the obstacles cause the 
controller 82E to calculate longitudinal and lateral risk components and provide appropriate 
allocation of various commands Fa, Fb and Fs to the longitudinal and lateral risks. 
Accordingly, the controller 82E allocates commands Fa, Fb and Fs to prompting operator to 
operating the vehicle longitudinally and laterally in accordance with the angular locations of 
the obstacles. 

[269] (7) The controller 82E provides an appropriate assist to manual effort to operate the 
steering wheel by modulating or altering steering reaction in response to the determined 
command Fs. Further, the controller 82E provides an appropriate assist to manual effort to 
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operate the brake pedal by modulating or altering brake pedal reaction in response to the 
determined command Fb- 

[270] (8) For each of the obstacles, the controller 82E calculates risk RPk to calculate 
longitudinal and lateral risk components RPungimdinai and RPLaterai: In response to the 
longitudinal and lateral risk components, the controller 82E allocates commands Fa, Fb and 
Fs for prompting operator to operating the vehicle 70E longitudinally and laterally in an 
appropriate manner to cope with such risk components. In this implementation, tiierefore, 
the driving system provides assist the operator in prompting to operating the vehicle 
longitudinally and laterally in response to distribution of individual risks RPk. In this 
implementation, the controller 82E determines RPk as a function of the reciprocal of TTCk. 
Therefore, the driving system assists the operator in prompting to operating the vehicle 
longitudinally and laterally in response to the extent how quickly the vehicle has approached 
the obstacle k. 

[271] In this implementation, the controller 82E uses the equation 19 to determine RPk as a 
physical quantity quantifying the extent how quickly the vehicle 70E has approached an 
obstacle k. The invention is not limited to the use of equation 19 to quantify the extent how 
quickly the vehicle 70E has approached an obstacle k. Any appropriate physical quantity 
may be used in quantifying the extent how quickly the vehicle 70E has approached an 
obstacle k. 

Ninth Implementation of the Invention 

[272] The ninth implementation is substantially the same as the eighth embodiment 
illustrated in Figures 38 to 47. 

[273] Turning back to Figures 38 and 39, Figure 38 provides hardware of the ninth 
implementation and Figure 39 illustrates how the hardware is arranged on an automobile 70E 
according to the ninth implementation. 

[274] In the ninth implementation, a microprocessor-based controller 82E recognizes the 
state of obstacles within environment in a field aroimd vehicle 70E by determining velocity 
Vf of vehicle 70E, relative position and relative velocity between vehicle 70E and each of 
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other vehicles within the environment, and relative position between the adjacent lane 
marking and each of obstacles within the environment. Based on the recognized obstacle 
state, controller 82E determines risk which each of the recognized obstacles would cause the 
operator to perceive. Controller 82E predicts a change m risk RPk in response to a change in 
operator input. Controller 82E calculates a change in total longitudinal risk in response to a 
change in operator input for longitudinal operation of the vehicle 70E and a change in total 
lateral risk in response to a change in operator input for lateral operation of the vehicle 70E. 
Based on the calculated changes in total longitudinal and lateral risks, controller 82E 
determines longitudinal and lateral commands. 

[275] Controller 82E inputs the information on the current operator input for longitudinal 
operation of the vehicle by detecting positions of an accelerator pedal 152 and a brake pedal 
154 (see Figure 39). Stroke sensors may be used to measure the positions of such pedals. 
The positions of such pedals may be determined from the state of a servo motor 146 and the 
state of a brake booster 1 50. 

[276] Controller 82 inputs the information on the current operator input for lateral operation 
of the vehicle by detecting manipulated angle of the steering wheel. 

[277] Controller 82E outputs the longitudinal commands for application to an accelerator 
pedal reaction characteristic modulator 144 and a brake pedal reaction characteristic 
modulator 148, respectively. In response to the applied longitudinal commands, the 
accelerator pedal reaction characteristic modulator 144 and the brake pedal reaction 
characteristic modulator 148 control servo motor 146 and brake booster 150, thereby 
modulating the accelerator pedal and brake pedal reaction force characteristics. Modulating 
the accelerator pedal and brake pedal reaction characteristics prompt the vehicle operator to 
manipulating the accelerator pedal 152 and the brake pedal 154 to appropriate positions, 
respectively. 

[278] Controller 82E outputs the lateral command for application to a steering reaction 
characteristic modulator 86. In response to the applied lateral command, the steering 
reaction characteristic modulator 86 controls servo motor 88. Modulating the steering 
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reaction characteristic prompts the vehicle operator to manipulating the steering wheel to an 
appropriate angular position. 

[279] The flow diagram in Figure 48 illustrates a control routine of the ninth 
implementation of the present invention. The control routine, generally indicated at 340, is 
executed at regular time intervals of 50 milliseconds. 

[280] The control routine 340 is substantially the same as the control routine 310 in Figure 
40 in that boxes 342, 344, 346 and 348 in Figure 48 are the same as the boxes 312, 314, 316 
and 318 in Figure 40, respectively. Accordingly, the detailed description on the boxes 342, 
344, 346 and 348 is hereby omitted for brevity. 

[281] hi Figure 48, after the box 348, the control logic goes to box 350. 
[282] At input box 350, the processor inputs information on current operator input through 
the depressed angle 9ab of accelerator pedal 152 and/or brake pedal 154 and through the 
manipulated angle of steering wheel. When the operator manipulates the accelerator pedal 
152, the processor uses the depressed angle of the accelerator pedal 152 as the depressed 
angle Gab- When the operator manipulates the brake pedal 154, the processor uses the 
depressed angle of the brake pedal 154 as the depressed angle Gab- 

[283] At the next box 352, the processor predicts a change in RPk in response to a change in 
operator input. Specifically, based on the current RPk determined at box 348 and the current 
operator input received at box 350, the processor predicts two new values of RPk when there 
is a imit change fi:'om the current operator input in both directions. 

[284] With reference to Figure 49, with regard to the operator input for operating the 
vehicle in the longitudinal direction, the processor predicts an increase in vehicle velocity 
when there is an increase by a variation AAB from the current accelerator/brake pedal 
depressed angle ^AB and it also predicts a decrease in vehicle velocity when there is a 
decrease by the variation AAB from the current accelerator/brake pedal depressed angle 
^AB. Based on the predicted increase and decrease in vehicle velocity, the processor 
calculates the new values of RPk using the equation 19. hi other words, the processor 
predicts one new value of RPk after the vehicle velocity imdergoes the predicted increase, 
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and the other new value of RPk after the vehicle velocity imdergoes the predicted decrease. 
Turning back to the equation 19, RPk is determined as a function of the reciprocal of TTCk 
that is determined as a function of relative velocity Vrk- From this explanation, it is clear that 
a change in vehicle velocity causes a change in vehicle velocity Vrk which in tum causes a 
change in RPk. 

[285] For ease of description, it is hereby assxmied that the accelerator/brake pedal 
depressed angle ^AB increases when the accelerator pedal 152 or brake pedal 154 is 
manipulated in a direction to cause an increase in RPk. For example, it is considered that 
there is an increase in accelerator/brake pedals AB when there is an increase from the current 
value of depressed angle of the accelerator pedal 152 or there is a decrease from the current 
value of depressed angle of the brake pedal 154. 

[286] With continuing reference to Figure 49, with regard to the operator input for 
operating the vehicle in the lateral direction, the processor predicts a change in heading 
direction of the vehicle when there is an increase by a variation AS from the current steering 
angle and it also predicts a change in heading direction of the vehicle when there is a 
decrease by the variation AS from the current steering angle 9S. Based on the predicted 
changes in heading direction of the vehicle, the processor calculates new angular locations of 
each obstacle k and new values of RPk. If there is a change in heading direction, there is a 
change in separation between the vehicle and each obstacle and a change in angular location 
of the obstacle. Accordingly, a change in steering angle causes a change in RPk. In Figure 
49, when the steering is in neutral position, dS is zero. 0S is positive upon manipulating the 
steering wheel to the right, and it is negative upon manipulating the steering wheel to the left. 
[287] The magnitudes of the variations AAB and AS are determined taking into account 
changes in pedal depressed angle and in steering angle per each processing time, which 
changes would normally occiu: during normal operation of the vehicle. In this 
implementation, the one processing time is, for example, from 0.1 to 0.2 seconds. The 
processing time differs depending on the kind of vehicle and on the type of processing to be 
completed. If the processing time ranges 0.1 to 0.2 seconds, AAB and AS are 10 mm and 0.5 
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degrees, respectively. The variances are not limited to such fixed values. By using learning 
technique, the average of depressed angles of accelerator/brake pedal and the average of 
manipulated steering angles over one driving cycle may be set as the variancesAAB and AS 
for the next driving cycle. 

[288] With reference again to the flow diagram in Figure 48, at box 354, the processor 
calculates a change in RPLongitudmai. Based on RPk determined at box 348 and its predicted 
new values determined at box 352, the processor determines the current total longitudinal 
risk RPLongitudmai(O), a ncw value of total longitudinal risk RPLongitudinai(+) for GAB +AAB, and 
the other new value of total longitudinal risk RPLongitnidinai(-) for ^AB -AAB. The current 
total longitudinal risk RPLongitudinai(O) is expressed as, 

[289] RPLongitudinalCO) = J) ^^^^ Eq. 23 

k 

[290] Using the equation 21, the processor determines the new value of total longitudinal 
risk RPLongitudinai(+) for GAB +AAB, and the other new value of total longitudinal risk 
RPLongitrudinai(-) for ^AB -AAB. Figure 50 illustrates the relationship between the current and 
two new values of total longitudinal risk. 

[291] At the next box 356, the processor calculates a change in RPLaterai- Based on 

RPk determined at box 348 and its predicted new values determined at box 352, the processor 
determines the current total lateral risk RPLaterai(O), a new value of total lateral risk RPuterai(+) 
for 0S +AS, and the other new value of total lateral risk RPLaterai(-) for OS -AS. The current 
total lateral risk RPLaterai(O) is expressed as, 

[292] RPLatena(O) = ^RP. sinfl Eq. 24 

Using the equation 22, the processor determines the new value of total lateral risk RPLaterai(+) 
for 0 S + A S, and the other new value of total lateral risk RPLaterai(-) for 0 S - A S. Figure 51 
illustrates the relationship between the current and two new values of total lateral risk. 
[293] At box 358, the processor determines longitudinal commands including a set of 

accelerator pedal reaction commands Fa(0), Fa(-) and Fa(+), and a set of brake pedal reaction 

commands Fb(0), Fb(-) and Fb(+). The processor determines accelerator pedal reaction 
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commands Fa(0), Fa(-) and Fa(+) by referring to the relationship illustrated in Figure 52 
using RPLongitudinai(O), RPLongitudinai(-) and RPLongitudinai(+), respectively. The fiiUy drawn line 
in Figure 52 is substantially the same as that in Figure 45. The processor determines brake 
pedal reaction commands Fb(0), Fb(-) and Fb(h-) by referring to the relationship illustrated in 
Figure 53 using RPLongitudinai(O), RPLongitudinai(-) and RPLongitudmai(+), respectively. The fully 
drawn line in Figure 53 is substantially the same as that in Figure 46. 

[294] At box 360, the processor determines a set of lateral commands Fs(0), Fs(-) and Fs(+) 
by referring to the illustrated relationship in Figure 54 using RPLaterai(O), RPtateraiC-) and 
RPLatcrai(+), respectively. The fully drawn line in Figure 54 is substantially the same as that 
in Figure 47. 

[295] At output box 362, the processor outputs longitudinal and lateral commands 
determined at boxes 358 and 360. These commands include a set of accelerator pedal 
reaction commands Fa(0), Fa(-) and Fa(+) for application to an accelerator pedal reaction 
characteristic modulator 144 (see Figure 38). They also include a set of brake pedal reaction 
commands Fb(0), Fb(-) and Fb(+) for application to a brake pedal reaction characteristic 
modulator 148 (see Figure 38). They further include a set of lateral commands Fs(0), Fs(-) 
and Fs(+) for application to a steering reaction characteristic modulator 86 (see Figure 38). 
[296] With reference to Figure 38, in response to the commands Fa(0), Fa(-) and Fa(+) 
from the controller 82E, the accelerator pedal reaction characteristic modulator 144 controls 
a servo motor 146 to modulate or alter the accelerator pedal reaction characteristic as shown 
in Figure 55. In Figure 55, broken line indicates the usual accelerator pedal reaction 
characteristic. At ^AB, the accelerator pedal reaction is increased from the leyel on the usual 
reaction characteristic by Fa(0). Upon manipulating the accelerator pedal from the current 
depressed angle toward^ AB - AAB, the operator feels reaction force increased from the level 
on the usual reaction characteristic by Fa(-). Upon manipulating the accelerator pedal from 
the current depressed angle towardGAB +AAB, the operator feels reaction force increased 
from the level on the usual reaction characteristic by Fa(+). From a change in reaction force. 
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the operator can recognize how the current risk will vary upon moving the accelerator pedal 
from the current position. 

[297] In response to the commands Fb(0), Fb(-) and Fb(+), the brake pedal reaction 
characteristic modulator 148 controls a brake booster 150 to modulate or alter the brake pedal 
reaction characteristic as shown in Figure 56. hi Figure 56, broken line indicates the usual 
brake pedal reaction characteristic. At 6AB, the brake pedal reaction is as high as the level 
on the usual reaction characteristic because Fb(0) = 0. Upon manipulating the brake pedal 
from the current depressed angle toward^AB - AAB, the operator feels the usual reaction 
force because Fb(-) = 0, Upon manipulating the brake pedal from the current depressed angle 
towardOAB +AAB, the operator feels reaction force decreased from the level on the usual 
reaction characteristic by Fb(+). From a change in reaction force, the operator can recognize 
how the current risk will vary upon moving the brake pedal from the current position. 
[298] In response to commands Fs(0), Fs(-) and Fs(+), the steering reaction characteristic 
modulator 86 controls a servo motor 88 to modulate or alter the steering reaction 
characteristic as shown in Figure 57. In Figure 57, broken line indicates the usual steering 
reaction characteristic. At ^S, the steering reaction is increased from the level on the usual 
reaction characteristic by Fs(0). Upon manipulating the steering wheel from the current 
angular position toward - AS, the operator feels reaction force increased from the level on 
the usual reaction characteristic by Fs(-). Upon manipulating the steering wheel from the 
current angular position towardGS +AS, the operator feels reaction force increased from the 
level on the usual reaction characteristic by Fs(+). From a change in reaction force, the 
operator can recognize how the current risk will vary upon turning the steering from the 
current position. 

[299] From the preceding description, it will now be understood that the eighth and ninth 
implementations provide the same features. However, the ninth implementation provides a 
feature which the eighth implementation does not. According to the ninth implementation, 
gradient of reaction force is created across the current position of operator input so as to 

WDC99 867822-1.050353.0598 

SUBSTITUTE SPECIFICATION 



60 



prompt the operator to selecting a desired change from the current position of the operator 
input to reduce the risk. 

Tenth Implementation of the hivention 

[300] The tenth implementation is substantially the same as the eighth embodiment 
illustrated in Figures 38 to 47, 

[301] Turning back to Figures 38 and 39, Figure 38 provides hardware of the tenth 
implementation and Figure 39 illustrates how the hardware is arranged on an automobile TOE 
according to the tenth implementation. 

[302] In the tenth implementation, a microprocessor-based controller 82E recognizes the 
state of obstacles within environment in a field around vehicle TOE by determining velocity 
Vf of vehicle TOE, relative position and relative velocity between vehicle TOE and each of 
other vehicles within the environment, and relative position between the adjacent lane 
marking and each of obstacles within the environment. Based on the recognized obstacle 
state, controller 82E determines individual risk which each of the recognized obstacles would 
cause the operator to perceive. Controller 82E sums all of the individual risks to give a total 
risk. Controller 82E determines an angular location in which the total risk originates. In 
response to the determined angular position, controller 82E determines gains to be applied to 
a total longitudinal risk and to a total lateral risk. Based on the gains and the total 
longitudinal and lateral risks, controller 82E detemiines longitudinal and lateral commands. 
[303] Controller 82E outputs the longitudinal commands for application to an accelerator 
pedal reaction characteristic modulator 144 and a brake pedal reaction characteristic 
modulator 148, respectively. In response to the applied longitudinal commands, the 
accelerator pedal reaction characteristic modulator 144 and the brake pedal reaction 
characteristic modulator 148 control servo motor 146 and brake booster 150, thereby 
modulating the accelerator pedal and brake pedal reaction force characteristics. Modulating 
the accelerator pedal and brake pedal reaction characteristics prompt the vehicle operator to 
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manipulating the accelerator pedal 152 and the brake pedal 154 to appropriate positions, 
respectively. 

[304] Controller 82E outputs the lateral command for application to a steering reaction 
characteristic modulator 86. In response to the applied lateral command, the steering 
reaction characteristic modulator 86 controls servo motor 88. Modulating the steering 
reaction characteristic prompts the vehicle operator to manipulating the steering wheel to an 
appropriate angular position. 

[305] The flow diagram in Figure 58 illustrates a control routine of the tenth 
implementation of the present invention. The control routine, generally indicated at 370, is 
executed at regular time intervals of 50 milliseconds. 

[306] The control routine 370 is substantially the same as the control routine 310 in Figure 
40 in that boxes 372, 374, 376, 378, 380 and 382 in Figure 58 are the same as the boxes 312, 
314, 316, 318, 320 and 322 in Figure 40, respectively. Accordingly, the detailed description 
on the boxes 372, 374, 376, 378, 380 and 382 is hereby omitted for brevity. 
[307] In Figure 58, after the box 382, the control logic goes to box 384. At box 384, based 
on the total longitudinal risk RPtongitudinai determined at 380 and the total lateral risk RPuterai, 
the processor calculates an angular location 0all in which a combined risk originates. The 
angular location0ALL is expressed as, 

[308] 0ALL - tan-^ Eq. 25 

When the combined risk originates in an angular location ahead the vehicle, 0 all = 0 degree. 
When it originates in an angular location behind the vehicle, 0 all =180 degrees. When it 
originates in an angular location on the right hand or left hand side of the vehicle, \^all\ = 90 

degrees. 

[309] At the next box 386, the processor determines gains GLongitudinai and GLaterai by 

referring to the illustrated curves in Figure 59 using the absolute value |^^| at box 386. In 
Figure 59, the horizontal axis represents the absolute value of </>all, while the vertical axis 
represents gain. The gain GLongitudinai is 1 when the absolute value \^all\ is less than a value <f)i 
that is less than 90 degrees. This is the case where the combined risk originates ahead of the 

WDC99 867822-1.050353.0598 

SUBSTITUTE SPECIFICATION 



62 

vehicle. The gain GLongitudinai drops from 1 toward zero when the absolute value \^all\ exceeds 
the value <j>i. At 90 degrees, the gain GLongitudinai is zero. This is the case where the combined 
risk originates form the side or from behind. 

[310] With continuing reference to Figure 59, the gain GLaterai is 1 at about 90 degrees. This 
is the case where the combined risk originates on the lateral side of the vehicle. At 0 and 180 
degrees, the gain CjLatrai drops down to zero. 

[311] At box 388, the processor determines an accelerator pedal reaction command Fa and 
a brake pedal reaction Fb in the same manner as the processor did at box 324 in Figure 40. 
As different from box 324, the processor multiplies GLongitudinai with each of Fa and Fb at box 
388. 

[312] At box 390, the processor determines a steering reaction command Fs in the same 
manner as the processor did at box 326 in Figure 40. As different from box 326, the 
processor multiplies GLateraii with Fs at box 390. 

[313] At box 392, the processor outputs the commands determined at boxes388and 390 for 
application to modulators 144, 148 and 86 (see Figure 38). 

[314] In the tenth implementation, gains GLongitudinai and GLaterai are determined for 
longitudinal commands and for lateral command. These gains are set in accordance with 
curves illustrated in Figure 50. With the use of such gains, the controller can allocate 
commands for prompting the operator to extensively coping with risks in an angular location 
where the combined risk originates. 

[315] While the present invention has been particularly described, in conjunction with 
various implementations of the present invention, it is evident that many alternatives, 
modifications and variations will be apparent to those skilled in the art in light of the 
foregoing description. It is therefore contemplated that the appended claims will embrace 
any such altematives, modifications and variations as falling within the true scope and spirit 
of the present invention. 

[316] This application claims the priorities of Japanese Patent Applications No. P2001- 
252422, filed August 23, 2001, No. P2001-389314, filed December 21, 2001, No. P2002- 
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138266, filed May 14, 2002, and No. P2002-177029, Filed June 18, 2002, the disclosure of 
each of which is hereby incorporated by reference in its entirety. 
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